Going beyond standard analytics: how do you measure the impact of platforms on adaptation policy, research and practice?

Background

The proliferation of portals and platforms sharing information online is expanding daily. This does not always result in a coordinated or systematic effort, which means knowledge is often fragmented and siloed leading to redundancy and/or replication. In an era when planning must accelerate to implementation and concerted climate action, we need faster ways to learn lessons from one another on knowledge sharing and exchange.  To support knowledge sharing and exchange between climate adaptation platforms, the Adaptation AGORA project has started this webinar series to engage climate adaptation platforms, encourage collaboration between platforms, and increase learning.

This webinar focuses on how climate adaptation platforms measure their adaptation impact, what innovative methods they use, and hopes to start the discussion around increasing and meeting the tangible impact the platforms are aiming to achieve.

The webinar aimed to:

• Discover innovative methods used by climate adaptation platforms to measure their adaptation impact.
• Share and learn from the experience of climate adaptation platform managers and owners.
• Provide a space for reflection and discussion about increasing the impact of platforms.

Agenda

The webinar, held on Thursday, January 23rd, was moderated by Rosie Witton and Sukaina Bharwani (SEI) and then featured an introductory presentation by Roger Street (University of Oxford) before moving into five presentations and a Q&A, followed by interactive discussion time. Presentations were given by different Climate Adaptation Platforms, including:

• Helena Karlsson, Swedish climate adaptation
• Antigoni Voudouri, AdaptiveGreece Hub
• Sophie Adam, KLiVO Portal
• Ana Lancho Lucini, Adaptecca
• Johnathan Cauchi, MIP4Adapt M&E Thematic Working Group

Key messages from speakers

Roger Street, presenting KE4CAP

• The Stepping-Up Knowledge Exchange between Climate Adaptation Knowledge Platforms (KE4CAP) project focused on providing a forum for platform teams to learn from each other and work together to address challenges. This involved looking at topics including supporting and working with local users, integrating cultural knowledge and capacities into climate adaptation platforms (CAPs), and building integration and coherence across CAPs.
• To support co-evaluation, learning and capacity development we need:
  • Web analytics and social media feeds that can provide useful insights
  • User’s feedback – if and why a platform and its services are useful, how they are being used, and what impact these ultimately have. Surveys (online and in-person); user workshops; steering committees; user reference groups can all be held to collect this information.
  • Representative user groups and user-driven governance structures – promote co-development, provide high-quality user feedback, enable user-testing and generate buy-in and uptake
  • Built-in tracking and feedback functions – facilitating monitoring, and continuous improvement
• However this is often inhibited by lack of resources including allocating funding (time) and capacity (expertise) for undertaking rigorous MEL – user engagement.

Helena Karlsson, presenting the Swedish Portal for Climate Change Adaptation

• Swedish Portal for Climate Change Adaptation was launched in 2007 and is managed by SMHI. The primary target group is municipalities and authorities, and the content is focused on tools, news, and case studies.
• Measuring impact is conducted by:
  • Analytics are used to monitor how people are entering the platform, where they go, and what they are interested in.
  • Website performance testing is conducted in addition to this to ensure its user friendly, and users have the option to provide feedback or ask questions.
  • Analysing media such as newsletter distribution so there are annual summaries.
  • Conducting surveys after training sessions and events to ask people about the website, what they want and how they use it.
• Helena shared a “sneak peak” of the updated website that will be launched in March 2025.

Antigoni Voudouri, presenting the Greek Climate Change Adaptation hub

• The Greek Climate Change Adaptation (GCCA) Hub has been developed within the framework of the integrated project “LIFE-IP AdaptInGR” by the Natural Environment and Climate Change Agency (NECCA) of Greece.
• The Greek Climate Change Adaptation hub plays a crucial role in supporting the implementation of adaptation policy and in raising awareness and momentum for adaptation action. It is expected to make a substantial contribution to the implementation of the National Adaptation Strategy. 
• The hub was launched in 2022, with a second version launched in June 2024 and a third version launched in December 2024.
• Measuring impact of the platform is undertaken through:
  • Analytics of the platform – measuring where users are accessing the content from e.g which country, how many users visit the platform, and when the content is accessed.
  • Building capacity of potential users of the platform is another method to help increase the impact of the platform e.g. conducting user trainings, guidelines, and frameworks.

Sophie Adam, presenting the KLiVO platform

• The German and Climate Preparedness Portal (KLiVO) was launched in 2018 after a comprehensive needs analysis and co-creation process, and is a governmental meta-information-platform, that guides users to relevant and verified climate services.
• KLiVO supports the federal government, the federal states, the district and municipal authorities and associations, as well as members of the public, in adapting to the unavoidable consequences of climate change, by providing helpful services offering guidance in taking precautions against the effects of climate change. The Portal brings together data and information on climate change and climate change adaptation services which can support users in dealing with the effects of climate change.
• To support and enhance the exchange between users and providers of climate services and for an ongoing co-development of KLiVO, there is an accompanying user-provider-network.
• Measuring impact:
  • Comprehensive needs analysis and co-creation process before the platform was launched.
  • Bi-annual analytics for the platform help to draw changes and comparisons to support in future development.
  • In-detail evaluation after 5 year to review the achievement of the objectives and gain insights about use and impact of the portal, in order to then draw conclusions for our further developments. Focused on user perspective and developed an analytical framework and impact model. The evaluation looked at measuring the impact through analytics and a market analysis of the current platforms landscape to identify KLiVOs unique selling point. A mixed method approach was used including open surveys and interviews. More information can be see about the evaluation here:
  • The real impact levels is always something difficult to assess and even the evaluation process conducted has limitations.
  • Important to discuss and learn from each other about how to measure impact.

Ana Lancho Lucini, presenting Adaptecca

• The AdapteCCa platform is a platform for exchange of information on impacts, vulnerability and adaptation to climate change facilitates coordination and transfer of information, knowledge and experiences in this field between the different Spanish administrations and the scientific community, planners and managers, both public and private, and other agents, allowing a multi-directional communication channel between them.
• The platform provides a common space to share experiences and knowledge about the impacts, vulnerability and adaptation to climate change in Spain and other parts of the world. It also supports the mobilisation of actors supports synergies with European initiatives and Europeans climate platforms to maximise complementarity and collaboration between platforms.
• Measuring impact:
  • Different approaches were used throughout different stages.
  • Firstly a baseline had to be established and this was undertaken by a consultancy in 2017.
  • Surveys and interviews were conducted to asses how many people were using the platform, what areas they worked in and how the platform is perceived. This was conducted in coordination with an evaluation of the national climate change adaptation plan. Information gathered helped identify areas to develop further on the platform.
  • Analytics is also collected on the platform to help measure impact and reach on social media, and for reporting purposes.

Johnathan Cauchi, MIP4Adapt M&E Thematic Working Group

• The mission implementation platform for climate adaptation runs a number of thematic working groups, one of which is the monitoring and evaluation to automatic working group.
• At present we have around six schematic working groups and basically financing climate action, citizen and stakeholder engagement, climate services, integrating medication and adaptation, transformative transformation, adaptation and of course monitoring and evaluation.
• TWG objectives are identified by the participants when the group is formed. The objectives of the M&E TWG include to identify common challenges to achieve monitoring, evaluation, reporting and learning for climate adaptation, and also to capture the various projects inside indicators, definition, metrics, and outcomes.
• Within the TWG there are a number of projects founded from Horizon 2020, Horizon Europe and the LIFE programme and there are around 33 project participants within the TWG.
• Outputs are produced from the TWG and will likely focus on challenges and lessons learned relating to M&E in the current projects participating in the TWG, so that this can be used to inform and support future projects. The outputs should be released towards the end of 2025.
  
  

The CAPs Webinar series. What’s next?

The Climate Adaptation Platform (CAP) webinar series builds on the discussions and potential collaborations initiated during the February 2024 webinar, which brought together EC-funded projects and CAPs to identify key challenges and opportunities for fostering synergies and collaborations between EC projects and CAPs, and also highlighted the importance of continuing these dialogues within the right types of forums and engaging a broader community.

The CAPs webinar series plans to continue regular meetings and plans for meeting at upcoming conferences. If you have any questions or are interested in participating in future events, please contact Rosie Witton (rosie.witton@sei.org) and Sukaina Bharwani (sukaina.bharwani@sei.org).