How to run your survey 2017-09-11T13:55:45+00:00

RUN YOUR SURVEY

We are committed to helping researchers collect useful management data for their projects and ensure the data can be aggregated into a comparable, global public good. Below you will find the process to join the global initiative.

The Development WMS is a detailed survey that allows the researcher to take an in-depth look at the managerial structures within an organization and systematically evaluate the quality of those structures based on a standard scoring grid. The cost of getting such detailed data is that the survey is not a simple instrument to administer and requires interviewers to undergo approximately 35 hours of initial training as well as ongoing training during the project.

With the goal of minimizing the cost of training and enable research teams to use this tool to its full potential, we have developed an online training course to help this effort. We detail here the usual process to get a survey started, and there is a link to get in touch with us with information about your project plan. We will get back to you to discuss your project and access to all the training videos and materials, as well as interactive calibration tools and the data upload portal.

Step 1: Plan and define your project

The project set-up can take anywhere between 3-6 months, depending on the project size, sampling frame availability and how easy it is to set up the infrastructure to carry out the project.

1

Sampling frame

The first thing you need is a sampling frame — that is, a comprehensive and exhaustive list of schools to sample from — and from there you can figure out the possible sample size. 

More Info
2

Sample size

Your total sample size will heavily depend on the total budget you have for your project as well as your target sampling strategy and on the size of the population universe.

More Info
3

Time frame

The time frame of the survey is a decision to be made by the researchers and tailored to the specific needs of the project. Previous projects have ranged between 3-6 months.

More Info
4

Mode of interview

The D-WMS can be conducted in person or over the phone. It is a choice that the research team has to make based on their local context. The data collected in both modes are comparable.

More Info
5

Infrastructure

Setting up the call center infrastructure is a crucial part of a phone project. Setting up the travel and data input infrastructure of a face-to-face project is equally crucial. Teams should plan ahead.

More Info

Step 2: HR and project organization

Once you have the fixed infrastructure in place — that is, the sampling frame, the hardware, the software and the physical space — you will need to hire the staff to carry out the project. We have experimented with a number of organizational formats and have found that one with the following structure is the best type of organization: one project manager, a set of supervisors (one per 4-5 analysts), and as many analysts as you need to complete the number of desired interviews within at most 10 weeks. You will also need a trainer to supervise the training week and remotely supervise and calibrate the team over time. We will also be there (remotely!) to help ensure the data collected is in line with the data quality of previous waves.

Project Manager

The project manager will be in charge of running the project. They will oversee and coordinate the work of the supervisors, compile the data weekly to send to the D-WMS team and liaise with the team to correct any data issues.

Example

Supervisors

The supervisors are in charge of 4-5 analysts, and their role is primarily to monitor at least 80% of the interviews of their team and ensure all analysts are properly calibrated. They also coach analysts and ensure targets are met.

Example

Analysts

The analysts are the people in charge of approaching managers (cold calling or in person), presenting the project concept and convincing them to take part in the survey. They need to be flexible, approachable and determined.

Example

Trainer

The trainer is a member of the wider D-WMS team who has extensive personal experience with the survey and coordinates the training week for supervisors and analysts. The trainer is hired on a short term consultancy contract.

D-WMS team

We are happy to help with your project! We can offer to commit our time to interested and serious teams with support such as carrying out ongoing data checks and coordinating with the trainer and PMs to ensure data quality.

Step 3: Carry out the project

Every week the team should cycle through the steps below.

Data collection round up

Every week analysts will have a set of completed interviews. Both analysts and supervisors should keep a list of completed interviews including the organization name, ID and date completed.

Supervisor “check off”

The supervisors are expected to review and sign off on every interview that their analyst team completes. Their team tally should match the sum of the interviews their analysts have recorded in their personal interview sheets.

Send data for verification

Every week, the project manager will collect all the interview score files from all analysts and send these files to the D-WMS team via the online portal for verification and feedback.

D-WMS team feedback

After data verification, the D-WMS team will send back a report to the project’s field team with a brief analysis of whether the data collection looks like it is progressing well and flag any possible issues that may arise.

REPEAT!

Repeat the process weekly until the end of the data collection phase.

A NOTE ON DATA OWNERSHIP

The project team who carries out the data collection will have full ownership of the dataset they are collecting. The project team will also have complete rights of first publication, and although the D-WMS team will have access to the data on an ongoing basis for verification, we will not use or distribute the data until the project team has had a chance to publish with the new dataset. The only intended use that the D-WMS team has for the data is twofold, below. If your team is interested in collecting data using this methodology but have an issue with either of these points, please let us know.

1For use in the benchmarking exercise that is open to all principals on this website: we would hope to be able to use the question average scores so principals who use our benchmarking tool can see how they stack up.

2For use in the anonymized global dataset: part of the larger purpose of this project is to create a large, comparable global dataset that can be used by researchers everywhere. Data would only be appended to this dataset after the data collecting team has published with their dataset and are happy to share it.

Ready to do this? Fill out the form and we will be in touch shortly.

LET’S DO IT!