Pakistan: External Final Project Evaluation

Project Description

City wide Partnership for Sustainable Water Use and Water Stewardship in SMEs in Lahore, Pakistan (WSP) is a three year (Jan 2013 to Dec 2015) project under SWITCH-Asia programme of the European Union. The aim of this project is to promote the adoption of sustainable consumption and production among selected small and medium enterprises (SMEs). The project SMEs are located in Lahore, Sheikhupura, Faisalabad, Kasur and Raiwind sites of Punjab. The project is funded by The European Union and being executed by three partner organizations i.e. WWF-Pakistan (lead role), WWF-UK and Cleaner Production Institute[1] (CPI). Lahore Chamber of Commerce and Industry (LCCI) and Small and Medium Enterprises Development Authority (SMEDA) are the associates of the project. The project targets selected SMEs of textile processing, leather tanneries, pulp & paper and sugar sectors located in Punjab.

The overall objective of the project is “By 2025 water efficient production and consumption predominates as best practice in Pakistan’s major industrial cities as part of a broad engagement of business in water management, contributing to improved environmental sustainability and poverty reduction within the context of sustainable development”.

The specific objective of the project is “By 2015, 300 processing and manufacturing SMEs in the target area have enhanced understanding of Best Water Management Practices (BWMPs), 75 high water using SMEs have increased water management capacity, and 25 SMEs are implementing BWMPs, supported by a multi-stakeholder city level water partnership.

The expected results (ERs) of the project are:

ER 1- Reduced water use and pollutant load: By 2015, 25 high water using SMEs in Lahore have reduced their water use by at least 15% and pollutant load by 15%, improving the sustainability of production at enterprise level, offering a business case for further engagement

ER 2 – Capacity building: By 2015, 75 cross sectoral high water using SMEs and supporting institutions in the target area have increased capacity to adopt BWMPs leading towards sustainable water management practices

ER 3 – Creating broad awareness for water stewardship: By 2015, further 300 SMEs in the region have enhanced understanding and knowledge of the impacts of unsustainable water use and recognize the wider community level benefits of water stewardship.

ER 4 -Multi stakeholder city wide partnership: By 2015, a multi-stakeholder city wide Water stewardship partnership , comprising SMEs, public authorities, supporting Institutions, MNCs and the Ravi Commission is supporting water sustainable production and consumption and facilitating better water governance.

ER 5 – Policy engagement and lesson sharing: By 2015, model replication is supported by provincial and national government and lessons are shared widely with policy makers and regionally through SWITCH- Asia Network Facility

Objectives of the Evaluation

WWF-Pakistan is seeking for an international consultant (individual/firm) for external final evaluation of its WSP Project. The project is funded by European Union under SWITCH-Asia Programme. The external final evaluation of the project has the following objectives

I. To independently verify project’s achievements as reported through progress reports and defined in the project’s log frame and consolidated work plan;

II. To assess the extent to which the project performed, this includes evaluating its relevance, effectiveness, efficiency, sustainability and impact.

Evaluation questions

The evaluator(s) should respond to the questions below.

Relevance

• How well the project was designed in terms of its relevance towards achieving the set objectives?


• To what extent did the project target and reach potential SMEs within specified sectors?


• How did the project respond to the needs of target SMEs, including how these needs were identified and addressed?


• What criteria were used for the selection of SMEs for the specified four sectors?

Effectiveness

• To what extents are the results that are reported a fair and accurate record of achievement?


• To what extent has the project delivered results regarding the reduction in water consumption, energy consumption and pollution load in ER1 SMEs.


• To assess the outcomes of training workshops conducted under the project


• What were the key drivers and barriers affecting the delivery of results for the project?


• To what extents the project’s partnerships were effective for the execution of the project?

Efficiency

• To what extent did WWF-Pakistan deliver results on time and on budget against agreed plans?


• To what extent did the project understand cost drivers and manage these in relation to performance requirements?


• To what extent has the project used learning to improve delivery?

Sustainability

• To what extent has the project engaged relevant stakeholders in the implementation of the project?


• Was any mechanism developed and implemented by the project which ensures that the benefits delivered by the project are sustained after project completion?

Impact

• To what extent and how has the project built the capacity of SMEs and supporting institutions in implementation of BWMPs?


• To assess social, economic, environmental, technical and policy level impacts of the project.

Required Qualification and Experience

The evaluator will be selected on the basis of the following criteria.

• An evaluation specialist with ten to fifteen years of experience in programme/project evaluation in an international development context.


• Experience of results-based monitoring and evaluation.


• Ability to design and plan the evaluation approaches including quantitative and qualitative research methods.


• Relevant subject matter knowledge and experience of water sector projects.


• Consideration of the extent to which the evaluator or evaluation team has appropriate knowledge/experience of working in Pakistan. This includes language proficiency to conduct the research required or that resources be made available (e.g. translator etc) to enable the research to proceed smoothly.

Deliverables

The final external evaluation report (draft and final version) should be no longer than 40 pages (excluding potential annexes).

The external final evaluation report needs to be a substantial document that

(a) Answer all the elements of the Terms of Reference (ToRs)

(b) Provides findings and conclusions that are based on robust and transparent evidence;

(c) Where necessary supplements WWF-Pakistan’s own data with independent research

The evaluation report should include the following contents

Executive Summary

Introduction

• Purpose of the evaluation


• Organization context


• Logic and assumptions of the evaluation


• Overview of project activities

Evaluation Methodology

• Evaluation plan


• Strengths and weaknesses of selected design and research methods


• Summary of problems and issues encountered

Findings

• Overall progress of the project


• Assessment of accuracy of reported results


• Relevance


• Effectiveness


• Efficiency


• Sustainability


• Impact

Conclusions

• Summary of achievements against evaluation questions


• Overall impact and business case of project interventions

Lessons learnt (where relevant)

• Project level – management, design, implementation


• Policy level


• Sector level

Recommendations

Annexes (such as)

• Independent final evaluation terms of reference


• Evaluation research schedule


• Evaluation framework


• Data collection tools


• List of people consulted


• List of supporting documentary information


• Details of the evaluation team


• WWF-Pakistan management response to report findings and recommendations

Time-Frame:

The total duration for the submission of final draft of the evaluation report will be four weeks(one week for planning and preparation, one week for field visits/meetings and two weeks for preparation of draft and final report).

Proposal Requirements

I. Individual CV

II. Organization profile ( in case of a firm)

III. Technical proposal (methodology to deliver consultancy)

IV. Financial proposal

V. Work plan

[1] CPI is a Lahore based National NGO