 |
Why undertake a Citizen Report Card Exercise?
The results of the Citizen report cards provides an insight into the
citizens perspectives on pro-poor services that are not always captured
in traditional information gathering exercises undertaken by institutions
or organizations. The views of the citizens are captured with regard to
issues such as availability and affordability of services from the different service providers. The method also differentiates between different groups of citizens; or even by geographic region, rural and urban residence, and level of household expenditure. This differentiation also has to be respected later on when the service providers design and implement improvements to their services. What is important is that the appraisals conducted using the Citizen Report Cards are augmented with expert assessments and service provider reports to obtain a comprehensive
picture of the strengths and weaknesses of service provision to various
groups. To this end, secondary data proves useful to complement and
crosscheck the data, if it is available.
Seven steps of undertaking a Citizen Report Card Survey
Step 1: Defining the scope of the Citizen Report Card analysis
- Clarity has to be achieved on the main issues and problems that one
wishes to examine. The more focused the problems and issues are the
better the information gathered will be targeted. Information is not
only of value for those undertaking the survey. It is also necessary to
determine what the citizens have to say about the problems and
issues being raised. The information augments actions and activities
being undertaken by others in the same field.
- Focus of the survey also has to be defined. The focus of the survey
encompasses either only a single service provider, several providers
or it can be even be used to compare different services providers with
each other.
- Purpose and use of the survey has to be agreed upon in order to
ensure that the survey can be focused properly. Within the purpose it
is also important to determine how the results will be used. If the
information is to be use for improving the services of the providers
then it is necessary that specific information is gathered needed for
this. Very different information would have to be gathered if the
results are to be used to improve development cooperation projects.
Figure 2: Seven steps for undertaking a Citizen Report Card Survey
- Data requirements and collection requires different methods to be
used. Of importance here is to determine the exact specific population
and sample size for the survey. The different possible types of surveys
to be undertaken need to be defined, whether random samples will be
undertaken or focus groups. This will also determine the skills that
the staff and field workers will require while undertaking the survey.
- Resource requirements for the whole exercise have to be thought
through. The length and the extent of the survey will be dependent
upon and determined by the financial and personnel resources
available. Setting the parameters that will be used in the sampling,
including sample size and method of interviewing will also dictate
the resource requirements. It is important to also reserve some resources
for contingencies that may arise.
Step 2: Defining the methodology
- The methodology used should reflect the information that one wishes
to gather. Consultations with key stakeholder groups in government,
private sector, civil society, and selected academic institutions are
undertaken in order to clarify both the purpose and use of the survey
and therefore the methodology to be used.
- Inputs of different stakeholders should be integrated into in the design
of the survey questionnaire. A balance has to be sought between quantitative and qualitative information. The citizen report card methodology is based upon quantitative methods that are enhanced through qualitative information derived from interviews and observations.
- By mixing the focus group techniques (qualitative approach) with
questionnaires (quantitative approach) the citizen report card not only
reduces bias, it also reveals errors in measurement. Furthermore, the
two approaches ensure data cross checks and verifications can be
undertaken and it helps to increase response rates.
Getting the “right mix” between quantitative and qualitative information
gathering is important.
- Qualitative methods generally provide information about citizens’
feelings or insights and the main techniques used include observation
and unstructured interviews. Qualitative approaches examine the
“feelings” and perceptions people have about service providers. The
approach often helps to clarify issues in advance of undertaking a
quantitative survey and can also be used to generate hypotheses.
- Quantitative methods usually are based more on numbers that can
possibly be statistically assessed. Correct sampling procedures, unambiguous questionnaires and ability to process and assess the data
using computer programmes are just some of the features of quantitative
methods. Quantitative approaches provide hard numerical facts on
how much was done where, when and by whom. The method is best
used when seeking to rank order answers to establish overall levels of
satisfaction with the services provided and to rank these and if the
answers are to be weighted in order to assist in decision making on
changes and reforms to be undertaken.
Step 3: Selecting the sample
- During the design stage of the citizen report card method the sample
also has to be determined. Scientific sampling allows the researcher to
make accurate inferences about a larger population, estimations and
testing of hypotheses. A decision has to be taken about the kind,
method or sample as well as about the size of the population. Information
needs, desired level of confidence and the available resources are
aspects that need to be considered while deciding upon the sample
size.
Sampling follows the following main steps:
- Defining the population
- Census of sample
- Sample design
- Sample size
- Opting for fixed or sequential sampling
Step 4: The questionnaire
- Questions should be focused and valid. Questions can be open-ended
or closed-ended (partially close-ended, closed-ended with ordered
choices and closed-ended with un-ordered choices).
- The instructions for completing the questionnaire have to be precise,
bias in the instructions; the wording of the questions or the choice of
responses should be avoided.
- The questionnaires should be pre-tested in order to assess the validity
of the questions and whether unexpected responses are possible.
- Ranking the questions in general order of priority (from most important
to least important).
- Questions should be clustered, which in turn can also be ranked.
- Clustered questions can follow a basic order: why, who, how and
what.
- Lastly, by placing the most interesting or important questions at the
start of the questionnaire the highest level of interest of the respondent
can be assured since it may gradually decline.
Step 5: Coding, pre-testing and analysing the results
- Coding quantitative questions in advance is practical and time saving
process that assists those undertaking the questioning from rapidly
filling out the questionnaires. Open-ended responses have to be
analysed and then a number attached to the response. To avoid
confusion it is advisable that one person undertakes the coding
exercise.
- Testing whether the questionnaire actually works is essential as part
of the overall quality control.
- Training the enumerators undertaking the survey is equally important,
short clear manuals and getting the enumerators to undertake
the testing all help to ensure that the process of actual data collection
is well done.
- Several techniques exist to analyse the data, including: averages, data
ranges, frequency and midpoint. Tables and graphs should be used to
present the data visually. Where spatial data exists (i.e. maps) data
should also be presented spatially on maps.
- Interpretation of the data is crucial it is at this stage the information
gathered becomes tangible. Bringing all of the information together
and presenting it in an understandable manner ultimately determines
the value of the information gathered. Different perspectives have to
be bought together and the issues described in an easily understandable
manner for the citizens.
Step 6: Making the results known
- Making the results known to a wide audience requires a well-documented
final report. “Popular versions” of the report containing the
key findings and recommendations for the masses should be publicised
in order to ensure wide circulation. Effective use of the media is
also necessary, including print, radio and television media coverage.
Media coverage helps in facilitating a broad public discourse about
what citizens think and what administrators should do.
- Typical contents for the final report includes: executive summary,
objectives, methodology, major findings, implications and recommendations.
Popular versions need to summarise the main findings and
recommendations. Use of simple graphics and tables in the popular
versions improves the readability.
Step 7: Perpetuating the exercise: Institutionalisation
In order to avoid the citizen report card being a single exercise a locally
appropriate approach for perpetuating the process has to be considered
from the very start. This is not only necessary in order to be able to repeat the exercise but is also of importance in gaining a feedback and checking whether the recommendations from the initial survey are being followed through. Gaining citizen feedback on the quality and availability of services can be of concern for: the citizens and civil society, the service providers themselves or a regulatory government institution.
While civil society may establish an own independent organisation to
continue or follow-up on the exercise, the service providers may undertake
the work through their own public relations office. On the other
hand, government may establish some form of “watchdog” institution.
|
top | 