Evaluation and Market Research
Evaluation and market research are core business for RobertsBrown. We take time at the beginning of each project to get the foundations right: to clarify the client's need for information and how the evaluation information will be used. This approach gives greater clarity to the evaluation and facilitates the choice of appropriate methods.
The right methods are the building blocks for a good product but good analysis makes the product useful. We do more than just report what people say. We analyse and assess the data to work out what it means and give the client meaningful and useful information.
For all research and evaluation projects, we recommend a strategic workshop with clients before writing a final report. The purpose of the strategic workshop is first to discuss what the results mean and second to assist the client to decide what strategic changes should be made.
Advice on Commissioning Evaluations
RobertsBrown also specialises in providing advice to clients about commissioning other consultants to conduct an evaluation. We have extensive experience in clarifying the evaluation requirements with projects; developing the statement of requirements; and developing criteria for selecting the best evaluator.
Qualitative research allows the client to develop a picture of the customer's experience and responses; and to understand the emotional reactions that often drive customer behaviour. It can also be used to identify important questions for a survey. Depending on the research goals we may recommend the use of projective techniques to gain insight into the details of personal experiences and the underlying thoughts and emotions. Or we may suggest the use of Category Instance Generation as a way of identifying and probing the short-cuts people use when decribing phenomena.
A properly conducted survey provides information about the extent of a phenomenon. We provide a high quality questionnaire design and data analysis service. We recommend that all questions be thoroughly pre-tested using peer-reviews and cognitive testing. We also recommend that the questionnaire be piloted to test sampling and the questionnaire structure. For large-scale surveys, we sub-contract specialists to do the fieldwork and provide descriptive statistics. We then analyse the results and interpret them for the client.
We have experience in using cognitive interviews both to test and refine survey questions and for in-depth interviews. In questionnaire design, cognitive testing elicits the respondents' interpretation of the questions; the types of memory recalled and the recall strategies used when responding to survey questions. In in-depth interviews, we use cognitive testing to test and flesh-out evidence: to identify and explore the short-cuts people use when talking about ev
Depending on the research goals we may recommend the use of projective techniques. Projective techniques may be used to test marketing material or to gain insight into underlying thoughts and emotions. Our use of projective techniques and our analysis of the data is based on both extensive experience and our own research into how the techniques work.
RobertsBrown also provide a technical review service, providing quality assurance to clients about the credibility of evaluations conducted by other parties. We have extensive experience in reviewing evaluation reports and commenting on the quality of the work.
This is another core business area. We have over 25 years experience working with Government and community agencies to develop performance-reporting frameworks and systems that meet booth the needs of the organization and Government requirements.
We work with program managers to articulate the logic of a program. A program logic models the way a program is intended to work and how it is intended to bring about change. A good program logic enhances understanding about what can be expected from a program. It also provides a sound framework for developing performance information and evaluation questions.