Professional Development Program Evaluation in Education
The participants are made up of 200 teachers from four different school districts. The survey will be offered to all of the participants. The survey will consist of 11 questions. A survey is a method of collecting information that can be conducted in person, through the mail, by phone, or through email. The majority of the survey questions will be closed-ended questions. Closed-ended questions are typically multiple choices and may include a scale that represents a range of possible responses. The survey will also include an open-ended question, which will allow the participant to respond with a textual response. Textual responses are normally asking the participant to explain the answer to another question in more detail. Another way to gain more detail is through a participant interview.
Of the 200 participants, only 10 percent will be interviewed (approximately 20 participants). Interviews are good ways to retrieve impact information about a program. The interview will be clear and focused and will represent a range of perspectives. Multiple perspectives will provide a level of research validity once results are determined to be similar across the same range. It is also very important that the interviewer creates and maintains a safe and trusting atmosphere with the participant. In some cases it may be helpful to match the interviewer with the participant based on age, gender, ethnicity, or language. The purpose of conducting an interview is to gather data that can be analyzed by the evaluation team.
The two types of data are referred to as quantitative and qualitative. Quantitative data typically comes in the form of numbers. In order to analyze the data, the evaluator must make four important steps: check the raw data, prepare the data for analysis, conduct an analysis based on the evaluation plan, and integrate the conclusions. In order for the data to provide adequate power of detection, at least 20 participants must be used for data collection. Twenty samples are enough to generate the tables of comparison and to rule out sampling size errors. Qualitative data is typically narrative and text-based. Qualitative data will come from the interviews and the open-ended questions on the surveys. It is important to determine which data findings are most relevant to the evaluation questions and the goals of the program.
There are many positives and negatives to both quantitative and qualitative analysis of data. For example, the strength of quantitative analysis is that the data is considered to be stable because it is made up of facts that represent reality. It is also objective, which means that the data is independent of people’s perceptions. The conditions are also more controlled and the results tend to be more reliable. The weaknesses of quantitative analysis are that the viewpoint is usually from the “outside” perspective and the data orientation comes from the predetermined hypothesis. Quantitative data lacks personality. Qualitative data analysis is constantly changing as the participants perceptions change. The strengths of qualitative data analysis include the discovery of subjective data. Subjective data are perceptions of the subjects within their environment. Another strength is that qualitative data is about the human person and can provide rich, natural, and dynamic findings. The weaknesses of qualitative data are that in some cases the reality of a subject is not very real at all. Also, in some cases it is unable to determine the complete picture of the findings. Focus can sometimes be blurred in qualitative data. Qualitative is human where quantitative lacks emotion.
As mentioned previously, it is important, as a rule, to have a sample of at least 20 people. Using the SPSS program will enable the researcher to create cross-tab tables to determine group comparison. T-tests and F-tests are also applicable SPSS functions. To improve the data collection, for each group there should be about 20 cases where any one case can represent one participant. The final task is to show tables and graphs that support the evaluation questions and goals of the program.
INTERVIEW PROTOCOL
Dear participants of the NSF-MSP Program:
I would like to take this moment to invite you to expand your answers that you completed on the survey. The survey was given to all 200 members. This interview is only being presented to 20 members. You have been chosen to participate in this interview protocol. This interview will only last thirty minutes. You will be provided with five questions. These questions will be subjective and will be open-ended. I will also be requesting your permission to tape record the session so that I may be able to pay better attention to your answers and also so that I don’t miss any important details. May I have your permission to record our session?____________________________________________(please sign)
Five questions:
1. Give examples of how you have increased collaboration in your classrooms?
a) did you think they would be effective at first?
b) Did the students respond well to collaboration?
2. What part of MSP has been the most beneficial to you professionally?
a) where do you see yourself professionally in ten years?
b) Do you think these programs helped you make a change?
3. What subject area do you think your students will improve upon based on your participation in MSP programs?
a) what subject area would you feel most comfortable teaching other than the one you are presently?
b) Do you think math or science will ever be reformed and taught differently?
4. What programs of the MSP do you think are most replicable and transportable?
a) do you think the summer activities should be presented during the same time and in the same manner?
b) What program represented for you the most fresh and new innovations?
5. What would you tell a new recruit about the NSF-MSP program experience?
a) do you think all new teachers should participate in the programs?
b) What one piece of advice would you offer an incoming new teacher?
