3. Characteristics of FSRP’s Participatory and Conflict Sensitive Impact Monitoring
The impact monitoring approach of FSRP shows the following three characteristics:
- Conflict sensitivity: Based on a thorough project related conflict analysis (see also method on conflict analysis) and the elaborated result chains of the project activities and outputs to achieve positive impacts at the beneficiary level. In doing so, FSRP tried to “do no harm” (refer to method on “Do No Harm”) but “do some good” by providing the beneficiaries with the capacities and opportunities to cope with the conflict and its consequences and to prepare the ground for sustainable conflict transformation and peace building. This could be ensured through the regularly monitoring of the following:
- The positive impacts (intended / unintended) of the project activities: Some of the intended positive impacts directly addressed either the root causes of the conflict (e.g. widespread poverty, caste and gender discrimination, unequal access to resources) or the consequences (e.g. economic deprivation, out-migration and the lack of trust within the communities). The project’s orientation towards “working on conflict” and the intended positive impacts of the project activities were identified as direct benefits in the result chains of FSRP.
- Possible unintended negative impacts of the project activities on the conflict during planning, implementation and monitoring activities.
- “Personal risks” to staff and beneficiaries (e.g. intimidation and harassments), “external factors and negative effects” of the conflict on the project (e.g. changes in policy and position by any conflicting party towards the project and its staff), as well as “sensitive issues” that could negatively affect the project’s image and acceptance (e.g. lack in transparency and impartiality in resource transfers, decision-making processes, selection of project area and beneficiaries).
- Participation: The main purpose of the strategy was to gather the necessary information through the active participation by beneficiaries during the project implementation. This was done by including the perceptions of the beneficiaries in a group based, interactive and participatory way. The participatory approaches helped to empower the beneficiaries and to develop their ownership of the project. The beneficiaries became aware of the progress and were able to assess the benefits and impacts of the project’s activities by analyzing the utilization of outputs themselves.
- Sound design: The approach and its methodological implications proved to be practical and handy and were easily adapted by the field staff and the beneficiaries.
4. Developing and Utilizing an Impact Monitoring Manual
In order to institutionalize and integrate the participatory and conflict sensitive approach into the regular project cycle and the daily work at different levels (management, district and beneficiary level) FSRP developed and applied an innovative impact monitoring manual, the process of which took the following four steps:
Step 1: Developing an impact monitoring manual, structured as follows:
- The first part of the manual introduces the basics of impact monitoring, and includes definitions of participatory monitoring and conflict sensitivity, the result chains of FSRP, an analysis of the conflict related to the project and the monitoring matrix (see figure 1).
- The second part of the manual describes the monitoring of positive impacts with detailed descriptions and procedures on formally collating data at beneficiary level.
- The third part of the manual describes the monitoring of negative impacts and the formal data collection at district level.
- The fourth part describes the monitoring of risks, external factors, and sensitive issues and the formal data collection at management level.
- The fifth part is a collection of annexes where all relevant data formats and checklists used for monitoring were compiled.
Step 2: Internalizing and fine-tuning the manual: After developing the manual, the manual and its procedures were internalized and applied by FSRP’s key staff. Necessary adjustments were made ensuring that the beneficiaries understood the process clearly and were able to actively participate in the evolution of the manual.
Step 3: Training and orienting field staff and beneficiaries on the piloted manual: FSRP provided training and orientation to responsible field staff on how to use the manual effectively. Likewise, beneficiaries were orientated to the monitoring approach, its procedures and their roles.
Step 4: Utilizing the impact monitoring manual based on the result chains: FSRP carried out bimonthly activity monitoring and six-monthly/ annual impact monitoring at district and beneficiary level (on beneficiary level primarily through monitoring workshops). The whole process of monitoring impact indicators involved the following elements:
- Formal and informal data collection: Formal data collections were conducted and information to be collected was discussed and documented wherever the data collection took place. Additionally, the staff also collected some data in an informal way by talking to the beneficiaries or other stakeholders everyday activities without the primary intention to monitor a specific indicator. If the information was important to the project the field staff verbally transmitted the information to the management team.
- Compilation and documentation of the collated data.
- Analysis of the compiled data.
- Evaluation, reporting and inputs into project steering.
Figure 1: The monitoring matrix
FSRP’s participatory and conflict sensitive impact monitoring approach was based on a matrix structure. Consequently, this practical example of a “Peace and Conflict Impact Assessment – PCIA” methodology follows the matrix rather than the stages described in the generic PCIA method (see references). The matrix structure depicts the links between:
- The subject of monitoring (monitoring of positive impacts; negative impacts; risks, external factors and sensitive issues),
- The level of monitoring (monitoring at beneficiary, district or management level), and
- The method of monitoring during different parts of the monitoring process (formal and informal data collection; data entry and compilation; analysis; evaluation and reporting).
The procedures for monitoring positive impacts are shown in the first column. The formal data collection was done at beneficiary level through monitoring workshops and field studies by staff members. At district level the collected data was documented, compiled and analyzed by staff. At management level, the compiled data from both districts was again analyzed and evaluated for project steering and reporting purposes.
The procedures for monitoring the possible negative impacts are shown in the second column. All staff members undertook informal data collection at the beneficiary level during their work by being present in the field and listening to beneficiaries. At district level, the data was collected in a more formal way, using guiding questions and checklists, the data was then documented, compiled and analyzed. At management level, the compiled data was again analyzed and evaluated for project steering and reporting purposes.
The procedures for monitoring risks, external factors and sensitive issues are shown in the third column. The data collection was done in an informal manner at beneficiary level and at local field staff level. All staff members were responsible for monitoring and communication information on risks, external factors and sensitive issues related to FRSP activities. Only at management level, the data was collected in a systematic way using guiding questions and checklists. This was due to the sensitivity of the collected information and likely security implications. Afterwards, the data was documented, analyzed and evaluated at management level for project steering and reporting purposes.
5. Monitoring Intended and Unintended Positive Impacts
5.1 Developing indicators for monitoring the intended positive impacts
The formal monitoring of the intended positive impacts took place at beneficiary level and was related to the 25 impact indicators developed by FSRP in its planning phase. Two types of indicators were distinguished:
- Quantitative indicators that can be counted and recorded easily by the staff (e.g. number of initiated and functioning self-help groups).
- Qualitative indicators that often needed to be perceived and realized by the beneficiaries themselves (e.g. improved feelings of security among the beneficiaries, improved caste equality). Thus, the staff and beneficiaries conducted the monitoring together.
For each indicator the personal responsibility, the methods (i.e. monitoring workshops, field visits and interviews, project records) and the timing (six monthly, yearly) were defined. Changes in the indicators were measured during the monitoring process and compared with a baselines and previous achievements. Figure 2 shows examples of FSRP’s indicators addressing some of the root causes of the conflict and its consequences.
Figure 2: Indicators addressing some root causes of the conflict and its consequences
||Method, frequency and responsibility
|7.000 poor and conflict-affected families have improved their self-sufficiency of food by 25 % (number of month in a year, in which the households possess food and generate income from their own production).
||Monitoring workshop with community based organisation supported by FSRP, biannually, District Management Team.
|50 % of the returnees (female and male internally displaced persons and refugees) are successfully integrated in their communities.
||Monitoring workshop with community based organisation supported by FSRP, biannually, District Management Team.
|Half of 7.000 poor and conflict-affected households are convinced that the project activities have contributed towards security and reduction of conflicts.
||Monitoring workshop in selected learning centres, every six months, Learning Centre Facilitator and Coordinator.
|Caste equality has increased.
||Monitoring workshop in selected learning centres and supported community based organizations, biannually, Learning Centre Facilitator and Coordinator/ District Management Team.
|At least 50% of all Learning Centres (non-formal education) promoted by FSRP serve as local problem solving platforms.
||Discussion at staff level, biannually, Learning Centre Coordinator.
|60 User Groups with at least 50 % women participating in village development measures.
||Data to be collected from the project records, annually, Social Mobilizer Coordinator.
5.2 Procedures for formal data collection
a) Data collection at the beneficiary level by staff
This type of data collection referred to quantitative indicators and methods. In general three methods were used to collect data:
- Collecting data from project records: Disaggregated data on beneficiaries’ output levels was collected from project records, for example, the number of trained participants disaggregated by caste, ethnicity, and sex.
- Regular field visits and interviews: Beneficiaries’ perspectives on the utilization of outputs (e.g., use of skills gained from training; increase in income from income generating activities, etc.) with simple field notes were used to record expressed views.
- Collecting data from district authorities: relevant data was collected from the District Development Committee (DDC) and other concerned line agencies on an annual basis (during the conflict the availability of relevant data was limited due to hindered access of government bodies to the project area).
b) Data collection at the beneficiary level by staff and beneficiaries
For the peace and conflict related impact assessments participatory monitoring approaches were applied which actively involved the beneficiaries, and gathered information on their perceptions of intended and unintended positive impacts. Therefore, monitoring workshops were organized every six months with selected community based organizations supported by FSRP. The participants spent time, i.e. one to two days or several evenings discussing all the relevant indicators of the project.
Project staff facilitated the monitoring process and the beneficiaries themselves measured the changes brought about by the project from their own perspective. Initially, staff oriented the beneficiaries on the indicators and the result chains (the link between input, output, utilisation of output, and outcome). In the workshops, three steps were followed for each indicator:
- The participants discussed the specific indicator and related changes they could observe/perceive;
- The participants judged these changes and tried to scale them;
- The participants then attempted to quantify how much the FSRP contributed to these changes.
Monitoring workshop usually consisted of three parts:
- Group discussions: The monitoring of each indicator started with a group discussion. On average at least twenty participants were invited for each workshop from different community based organizations. The group discussed and monitored all relevant indicators during the workshop with participants being guided by the following key questions:
- What has changed (regarding each indicator) within the last six month, or since the start of the project? and;
- Which changes (in terms of the indicators) and at what level (individual, family, community) have changes been experienced?
Examples from previous monitoring periods were usually presented at the workshop as reference and participants were then asked to analyze the differences occurring across the monitoring periods. In addition to the key questions, it was often helpful to ask some further guiding questions such as:
- “What do you understand under this indicator?”,
- “Can you give some examples?”,
- “How was it before – how is it now?” (see also figure 5)
In ending the group discussion, the participants proceeded to scale the described changes in the indicators.
- Scaling the changes: The intended purpose of scaling the described changes was to compile all the positive and negative examples and impacts on one scale. Figure 3 and 4 show two types of scales that were used to measure the changes using either ordinal numbers or percentages. The key question used for scaling the changes with ordinal numbers was as follows:
- “How much did the situation regarding this indicator change in the last six months, or since the start of the project?”
- The second type of scaling changes used percentages and was only used for the monitoring indicators, which included a percentage number (e.g. 50 % of the returnees were successfully integrated into their communities). The following key question was usually asked:
- “How much did the situation regarding this indicator change from the beginning of the project until now (in terms of a percentage)?”
Because of the high levels of illiteracy amongst beneficiary group members, a visual scaling approach was applied. This assisted monitoring workshop participants in attempting to quantifying the changes they had experienced (negative, positive, no changes). Following an explanation by the monitoring workshop facilitators, each participant was asked to individually place its mark on the value of the changes they perceived to have occurred during the period being covered by the monitoring workshop (usually six months). Following this, the participants as a group were asked to give feedback on what they saw as a group and to reach a consensus on an overall score. If no consensus could be reached, the FRSP staff could still extract an average change as expressed.
Figure 3: Scaling the changes with ordinal numbers
Figure 4: Scaling the changes with percentages
Photo 1 and 2: Scaling the change with ordinal numbers
Scaling the contribution of FSRP: Once the change of the various monitoring indicators had been scaled, participants at monitoring workshops were asked to quantify the contribution of FSRP that may have influenced the changes (in percentages term), again using a scaling approach demonstrated in Figure 4. This also assisted FSRP in identifying external factors at play in the project areas
Figure 5: Example of monitoring the indicators
6.1 Assessing unintended negative impacts when planning project activities
To assist in the assessment of possible negative impacts within the social feasibility studies during the planning process, FSRP applied the following three steps:
Step 1: Assessing possible negative impacts:
The following questions were used to identify possible negative impacts from FSRP at the initial phase of planning its activities:
- Could FSRP’s activities exclude certain groups or place them at a disadvantage?
- Could FSRP’s activities support divisions, conflicts, disputes, or do harm to anyone?
- Could FSRP’s activities endanger human security?
- Could FSRP’s activities reach and address the poor and most vulnerable?
- Could FSRP’s activities address the interests and expectations of both conflicting parties?
Step 2: Selecting important and relevant possible negative impacts:
Identified negative impacts were discussed and analyzed and ranked utilizing the following guiding questions:
- Which possible negative impacts are most relevant and likely to occurring?
- Which are relevant to this conflict?
Step 3: Documenting possible negative impacts:
Negative impacts that were assessed as relevant, important, and related to the conflict were documented and integrated into the regular monitoring process. For this purpose, checklists and guiding questions were developed as permanent working tools consisting of questions such as:
- “Do our project activities create suspicion to one or both conflicting parties”?,
- “Do the government institutions feel weakened by the FSRP activities”?
- “Do the State Security Forces/ insurgent forces suspect FSRP of transferring resources to the “other side”?.
6.2 Assessing unintended negative impacts while implementing project activities
Periodically, staff members observed and monitored GTZ-FSRP’s activities for the likelihood of negative impacts upon the conflict. In identified urgent cases, the project reacted immediately. Generally staff members collected relevant information in an informal manner in the field, then discussed and documented more formally during staff meetings using special reporting formats for ease of interpretation and risk assessment.
The following steps were applied:
Step 1: Assessing possible negative impacts using guiding questions
(as in step 1, in 6.1 above)
Step 2: Discussing possible negative impacts using key questions such as:
- Did FSRP’s activities in general produce any negative impacts in the last month(s)?
- Did FSRP’s specific outputs produce any negative impacts in the last month(s)?
- Which negative impacts have been observed at district level?
- Are these negative impacts relevant and important?
Step 3: Analyzing the consequences of the assessment of possible negative impacts using the following categories:
- No reaction: There are no negative impacts or they are not relevant or important
- There is no need for action.
- Alert: There are negative impacts, which are relevant and important
- Document and continue to observe the situation carefully.
- Discuss this issue at management level in the next management meeting.
- Action: Negative impacts are occurring, which are relevant and important.
- Actions to be taken according to the situation (strategic, personal, operational):
- Immediate action is required for issues that can be solved at district level.
- Consult and coordinate with the management team for issues that need approval from the project’s team leader.
- Always inform team leader and management team.
7. Monitoring Risks, External Factors and Sensitive Issues
For monitoring different types of threats, FSRP developed three types of inquiring checklists which were utilized in the field. In every checklist, several questions were asked related to so called “personal risks” to staff and beneficiaries, “external factors and negative effects” of the conflict on the project, and “sensitive issues” that could negatively affect the project’s image and acceptance in the field.
- The “personal risks” checklist focused on the possible risks to the project staff and beneficiaries and included questions such as:
- “Do the staff or beneficiaries feel insecure”?
- “Is there evidence in incidence of harassment and intimidation (like forced donations)? and is this changing”?
- “Are there any negative attitudes towards our staff or the beneficiaries and if so by whom and why?”
- Are there any significant operations by Security Forces and/or Maoists in the project area that endanger our staff and the beneficiaries (e.g. searching operations, patrolling, crossfire, attacks)?
- The “external factors and negative effects” checklist focused on the impacts of a changing conflict situation on the project and included question such as:
- “Did one of the stakeholder/conflicting parties change their policy towards the project”?
- “Did one of the stakeholders change their position towards FSRP after repositioning of personnel at decision-making level?”
- “Are there any strikes or blockades with negative effects on project activities?”
- “Did any stakeholder (including conflicting parties) try to interfere on material transport and resource allocations by the project?”
- The “sensitive issues” checklist focused on identifying any emerging negative trends or examples of violation of FSRP’s working principles on participation, communication, transparency, impartiality and equality, thus, possibly creating divisive factors and tensions. Questions were used such as
- “Are there any negative examples in the transparency of the project such as
- The use of resources?
- In decision-making?
- In the selection of beneficiaries and staff”?
- “Are there any emerging trends of unequal participation in project activities”?,
- “Are there any cases of gender and caste inequality in terms of project allocation, wages, decision-making?
- “Are there any emerging cases or evidence, which could challenge FSRP’s political impartiality”?
- Are there any negative trends in the communication to the beneficiaries and other stakeholders?
The results from the application of the checklists were analyzed at staff and management meetings and judged to be either:
- Not important,
- Not serious but relevant for reporting,
- Serious and necessary to observe and discuss at the next management team meeting, or
- Very serious and in need of immediate action.
The checklists were evolved and integrated as permanent working tools with regular review meetings at management level to ensure they stayed focused.
8. Lessons Learnt
Due to its participatory and conflict sensitive approach the FSRP not only ensured the conflict sensitivity of its project activities through a process of continuous reflection on the interactions between the project activities and the dynamics of conflict, but also through the effective project steering while working in and on conflict. Therefore, the conflicting parties appreciated this approach of monitoring and reporting, thus further increasing the acceptance of the project in Rukum and Rolpa districts. This strengthened the projects political impartiality.
Advantages of the conceptual design
- The “do no harm but do some good” approach was incorporated into regular monitoring and steering activities (monitoring of the positive and negative impacts).
- Issues of risk assessment and management were interlinked with the monitoring of impacts: monitoring “personal risks” and “sensitive issues” that could negatively affect project’s and staff’s image and reputation, as well as “external factors and negative effects” of the conflict on the project.
- A systematic exchange of information at different phases of the project cycle and at different working levels between beneficiaries, staff and management was ensured.
Advantages of the participatory approach
- Ensured effective communication and feedback: Active participation of the beneficiaries ensured that the concerns of the communities, their opinions and perspectives, as well as their right to know about project activities, output, utilization, impacts and benefits were analyzed by themselves. The participatory approach increased the mutual respect between the project staff and the communities and became an effective tool for communication and feedback loops between the beneficiaries and the project.
- Accurate steering: The involvement of beneficiaries in the monitoring process helped to assess and gauge the impacts of the project more accurately than monitoring carried out by staff alone. This, indeed, helped to identify gaps in planning and implementation as reference for future projects. Furthermore, timely and accurate adjustments and redesigns of activities according to the interests and needs of the beneficiaries took place and unknown unintended impacts were identified and recognized.
- Self-realization process: Due to the beneficiaries' involvement in project monitoring and steering they became more aware of the benefits and gaps of project activities. This self-realization process helped them to understand the complexity of change processes, which also enhanced their self-help capacities.
- Transparency and ownership: As the monitoring process was transparent and did not require external involvement, it also helped the community to develop confidence and a feeling of ownership over the projects. This led to increased participation of beneficiaries in the project activities overtime and ensured continued monitoring even during periods of escalating conflicts. This ownership enabled the communities to advocate for the activities of the project to the conflicting parties.
- Empowerment of disadvantaged groups: Women and other excluded groups were included and got a chance to actively participate in the monitoring processes, which further empowered them. Scaling tools were used by the communities to measure differences in attitudes towards gender and caste discrimination.
- Credibility towards donors: The monitoring approach helped to inform and convince the donors of the effective utilisation of their investment. The communities themselves could provide examples and pointed out events resulting in positive changes.
- Monitoring occurred every six months, which was too short a time-span to measure social changes effectively and accurately.
- Difficulties were encountered in compiling the scaled data in the monitoring process. This led to difficulties in measuring the exact achievement of the project at district level during some of the time periods.
- The monitoring system was mainly based on qualitative information.
- Not all participants at the monitoring workshops understood the methods of scaling the changes in ordinal numbers and percentages.
- On occasions, representatives of the selected community based organizations carried out the scaling, which may not have fully reflected the perspectives of the whole community involved.
- Participants were sometimes biased when scaling achievements because they were afraid that – if the figures were too negative – support would stop and or not be provided in the future.
- Participants tended to evaluate the impact of the project in general terms, and rated the project’s contribution as positive. Culturally, it is rare and unlikely for a typical Nepali to pinpoint negative impacts of a project. Furthermore, external factors might also have influenced the opinion of the beneficiaries such as ongoing military operations.
- During the war the use of words such as conflict, security, Maoist, etc. were not allowed by the Maoists so that the staff had to use substitute words for these terms. This may have led to some misinterpretations of words during the monitoring process.