Monitoring, Evaluation and Research Paper

Introduction

Reliable data and critical joint reflection are crucial for making decisions about where to invest, how to sustain and improve water and sanitation services, and for understanding, which policies and strategies work. And yet widespread lack of capacity for monitoring, inconsistent or fragmented gathering of data, and limited use of information management systems continues to impede effective decision-making in sector planning, resource allocation and policy development. Most developing countries, the need for M&E is initiated by outside institutions like multinational agencies, development banks and donors, to promote improved measurement, monitoring and management for result . In recent years there has been a growing push for developing government- or country-led M&E systems in developing countries . There are a number of advantages to this over donor led project-based M&E systems and in the contexts of WASH and CCA the need may be even more pertinent. Government-led monitoring is often favored because it is believed that if countries are in charge of M&E processes, findings will better reflect the information needs and values of country stakeholders rather than just those of donors. Monitoring and evaluation also helps “improve budgeting, decision making, inter-governmental fiscal control, enhance the quality of government policy and end corruption . In most developing countries, there is a move toward a Results-Based M&E system, as it “provides crucial information about public sector performance, a view over time on the status of a programe, promotes credibility and public confidence” . The movement towards public sector reforms was propelled by a growing trend towards democracy, accountability and transparency, which has placed substantial emphasis on the M&E of development and tracking of public resources .

Inappropriate design, poor quality of construction and a lack of follow up and maintenance arrangements were critical issues present in all countries without exception. These were often not only the perceived reasons for failure, but also a deterrent to participation in subsequent projects. Several countries in the region are prone to natural disasters like cyclones and floods, yet the toilets here are not often designed to withstand or survive such calamities; as a result, communities that had begun to use toilets are often forced to revert back to open defecation. Poor design and poor quality of construction are often a result of a lack of supervision during the construction and implementation phase. In other cases, although projects are designed and implemented well, they quickly become unclean and unusable due to poor or no maintenance, or a lack of water. In India, maintenance was perceived to be the primary reason for the failure of many projects, especially for community facilities like toilets. Communities also refused to use incomplete or damaged toilets. In Sri Lanka too, technical quality and inappropriate designs have plagued the sector. Projects in the Tsunami affected areas are the most glaring examples of poor quality work.

Country Overview of M&E

Monitoring and Reporting (M&R) section has been newly established in the Ministry of Rural Rehabilitation and Development (MRRD). The M&R section has developed monitoring format with the support of Water and Sanitation Group and other INGOs. The Management Information System (MIS) and Geographical Information System (GIS) system have been newly established and launched. The system is accessible in MRRD website and relevant sector partner can feed the required information to the system and vice versa can retrieve the required information from the system. The system also has started to collect and disseminate disaggregated data at the regional level.
The statutory responsibility for the sanitation sector vests in the Ministry of Local Government, Rural Development & Cooperatives (MoLGRD&C). Monitoring is one of the key challenges for Bangladesh for implementing the water and sanitation related targets in addition to Integration in the national planning process; financial and non-financial resources allocation; institutional mechanism for implementation and participation and accountability. However, there is realization to conduct nationwide periodic monitoring, particularly in the Hard-to-Reach areas with low sanitation coverage to guide the decision making for achieving Open Defecation Free (ODF) Bangladesh at the national level and to understand specific gaps and investment needs. In Bangladesh, after interventions were phased out it was found that many of the hygienic toilets constructed were no longer hygienic because of broken slabs, and seats, broken water seals, and filled up pits due to the poor quality of construction. In addition, the design and technology itself was perceived to be inappropriate for specific locations. People pointed out that the geophysical characteristics of the coastal inland saline areas, coastal islands, and hilly areas were all different, calling for different sanitation technologies. A good example in technology innovation is the development of an improved “seal” for hygienic pour-flush latrine pans through partnership among the Bill and Melinda Gates Foundation (BMGF), International Development Enterprises (IDE), and World Bank's Water and Sanitation Program (WSP), coordinated by the DPHE. The pan incorporates a counterweighted trap door that swings to the open position upon rapid addition of less than half liter of water. However adequate funding and efforts around research & development, documentation and dissemination remain as challenges for technological advancement.
Annual household survey is an institutionalized regular activity within the Ministry of Health (MoH). Sanitation and hygiene related data are collected annually by the health workers and compiled, analyzed and reported. However the existing monitoring system in place provides information for basic sanitation facilities only. New indicators that reflect the hygienic toilet usage and hand washing with soap have been incorporated into the 2015 Bhutan Management Information System. The Royal Government of Bhutan’s (RGoBs) 11th Five Year Plan for 2013-18 has prioritized sanitation and hygiene as a key sector result area. Local government is responsible for setting targets, developing initiatives, and identifying and monitoring progress indicators aligns with the decentralization process outlined in the Local Government Act 2009.
India has upgraded the Integrated Management Information System (IMIS) to maintain a register of all rural households in the country, along with their sanitation status as recorded in the baseline survey carried out in 2013 . The Ministry of Drinking Water and Sanitation (MDWS) are managing the IMIS. A Mobile application with geo tag features for uploading photographs of toilets has been launched on October 2, 2014. Third party monitoring, which is doing by the National Sample Survey Organization (NSSO). A module for monitoring Open Deification Free is also available on the IMIS. Guidelines for ODF verification have also been issued, and the States have begun the process of verification. Further, the IMIS provides for a module, whereby, the States/districts can transfer incentive fund directly to the community as a whole, as per the flexibility provided to the States. Toilet usage and sustainability monitoring is being done through a third party mechanism of Rapid Action Learning Unit (RALU). RALU is also doing impact monitoring through a health outcome based study. Community based monitoring is also developed to ensure ODF status. In addition, Monitoring is one of building block for new program like Swachh Bharat Mission (SBM) programme. Third party monitoring through agencies such as National Sample Survey Organization (NSSO) is also being carried out.
The draft Water and Sanitation policy of 2015 takes note of the constitution and emphasis to improve governance by developing human resource capacity, promoting awareness, strengthening monitoring and evaluation, research, and learning at all levels. However, One of the major challenges in water and sanitation includes lack of stakeholder participation in the design and monitoring of water and sewerage systems.
The Ministry of Urban Development (MoUD) is the lead Ministry of the WASH sector. The Department of Water Supply and Sewerage (DWSS) under MoUD, with its regional and district offices, is the lead agency of the WASH sector to facilitate the execution and monitoring of the WASH sector projects nationwide. The MoUD has formulated a WASH sector monitoring and evaluation framework in 2009 with key sectoral indicators. The framework has proposed nine CREAM (Clear, Relevant, Economic, Adequate and Monitor able) indicators, which include: safe water, accessible water, adequate water, access to improved sanitation, equity, improved health, improved work/economy and sustainability over time. Planning, Monitoring and Foreign Aid Division of the MoUD is responsible for facilitating and periodically updating progress and performance of the sector. A dedicated Monitoring and Evaluation unit under the same division is responsible for collecting and disseminating information on a regular basis. National Management Information Project (NMIP) under DWSS provides data feedback for sector planning and monitoring. Besides, Department of Education (DoE) and Department of Health Services and other concerned ministries and stakeholders have considered sanitation and hygiene as one of the key progress indicators in their development frameworks.
Pakistan decentralized WASH to the provinces in 2011 after 18th constitutional amendment. Since then role of monitoring has also been relegated to the provinces with respective service delivery institutions i.e. Public Health Engineering, Urban Development and Local Government Departments in different provinces. At the provincial level, there is a directorate for monitoring and evaluation called Directorate General Monitoring and Evaluation (DGME). The DGME may have offices at divisional level, but does not have enough presence at district level. However, WASH is not fully integrated with DGME framework, and its monitoring is driven by service delivery section so process and outcome monitoring is not fully realized. WASH service providers should also work with other stakeholders like provincial bureau of statistics and information technology department who are responsible for gauging the progress and strengthening the management information systems for informed decision making. There is need to work on creating uniformity and consistency in the households surveys conducted at the national and provincial levels for tracking the outcome and coverage of WASH services. In Punjab, the health related surveillance is being linked with WASH service providers for timely actions.
National Water Supply & Drainage Board (NWSDB) under the Ministry of City Planning & Water Supply (CP&WS) has the mandate for the development of water supply, sanitation and sewerage services in urban as well as suburbs. The newly established Department of National Community Water Supply also is playing a key role in rural areas. The Ministry acts as the national body for monitoring and review of sector activities. Ministry of Health is responsible for water quality monitoring and Ministry of City Planning and Water Supply is responsible for Sector Policy, coordination, budget allocating, and monitoring and review of progress. According to MGD report 2015, Sri Lanka has achieved the MDG goals in water and sanitation. The NWSDB established a forum for monitoring MDGs represented by key stakeholders in the sector including the Department of Census and Statistics. In 2013, an agreement was reached with DC&S to include the indicators for national survey for water and sanitation in line with JMP indicators. The forum contributes to national efforts in implementing/ improving sanitation services in urban, rural and estate sectors for reporting on achievements. In order to reflect uniformity in national and international monitoring and reporting, it has been proposed to have an independent national body comprising key stakeholders to monitor WASH coverage based on agreed indicators. This idea is strongly advocated in the SGD monitoring phase. In addition, gender monitoring and evaluation indicators in targeting SDG goals have been adopted. SDG targeting and monitoring in the WASH sector with an independent expert panel has been mooted in many forums to overcome discrepancies in reporting progress.

Country Overview of M&E

According to the Global Learning Assessment and Analysis of Water and Sanitation (GLAAS) report 2016, Afghanistan remained on the top in terms of availability of data for decision-making for water supply related to; policy and strategies, national standards, resource allocation; status and quality of service delivery. India remained second with having data for policy and strategies; national standards and resource allocation. The rest of the countries either limited data available or not sufficient data available for decision making. For details of member countries on data availability for decision-making is provided in the table below.
 Drinking-water
Policy and strategyNational standardResource allocationStatus and quality of service delivery
Afghanistan
Bangladesh
Bhutan
India
Maldives××
Nepal
Pakistan
✓Data available, analyzed and used for a majority of decisions
× Data available but not sufficiently use for decision-making
◾ Only limited data collection and limited availability
Source: GLAAS report 2017
Regarding sanitation, both India and Afghanistan have data available, analyzed, and use for a majority of decisions i.e. policy and strategy, resource allocation and status and quality of service delivery. The rest of member countries have limited data available but not sufficient for decision-making. For details of member countries on data availability for decision-making is provided in the table below
 Policy and strategyResource allocationStatus and quality of service delivery
Afghanistan
Bangladesh
Bhutan
India
Maldives×
Nepal
Pakistan
✓Data available, analyzed, and use for a majority of decisions
◾Data available but not sufficiently used for decision-making
×Only limited data collection and limited availability
Source: GLAAS report 2017

Overall the member governments initiated to establish monitoring mechanism to track the progress on water and sanitation sector. However, India and Afghanistan have established management information system, which is accessible to every stakeholder to take inform decision. Other member countries need to establish nationally agreed and developed information systems. For innovation, the goal is to seek affordable, effective, and hygienic sanitation technology solutions that can improve the quality of sanitation services for the billions of people currently using non-piped sanitation systems. Further, South Asian countries have not utilized the technological options and innovation as integral component of its service delivery though adhoc examples exist.

  1. How to actively engage multi-stakeholder in reflection and learning processes that can transform monitoring information and sector experience into actionable lessons for the sector? How to demonstrate flexibility and openness to adjust policies, strategies and practices, based on sound evidence, good practices and innovations?
  2. What are the lessons learned and best practices? Are we able to align project planning with evidence-based researches? Is it linked with audits?
  3. What is project monitoring, how it has evolved in public sector? Are current systems of monitoring adding value to quality of services?
  4. How to develop result based monitoring framework and systems for WASH aligned with SDG targets.
  5. Do we have clarity on monitoring indicators to mainstream the gender, health and nutrition as one of the key gauge for social impacts of WASH programs?
  6. What role technology and private sector has to play for ‘safely managed’ in SDGs. What steps countries can take to promote innovations and technology choices (including from proactive solutions presented by private sector) for meeting the aspirations in SDGs?
  7. What technological options with low O&M costs are available for fecal sludge management? Do we have options for reuse of waste to generate energy?
  8. Are we working on appropriate sanitation solutions for flooded zones (e.g. communities that face seasonal flooding, high groundwater tables, riparian or tidal communities, floating communities, etc.? Technologies should improve upon or develop new sanitation technologies that cope with wet-environment conditions?
  9. Are there any lesson learned/best practices? Why pilot /small-scale model fails when scaled up? Why we have not been able to learn from each other? Our cross learning approach is limited to exposure visit?