Program Evaluation

Group Members	Roles	Commented On
Kristi Burkhart	Program Evaluation	Group one - Experiential learning
Allison Hillis	Co-leader,table, introduction	Group two - Narrative learning
Mallory McDonald	Summarize responses	Group two - Narrative Learning
Danielle Riddell	Reflection	Group two- Narrative learning
Shannon Staller	Co-Leader, Program Evaluation	Group One-Experiential leaning

Program Evaluation: Developmentally Appropriate Practice in Early Childhood

Kristi Burkhart, Allison Hillis, Mallory McDonald, Danielle Riddell, Shannon Staller

Ball State University

Introduction

            Our program is designed to support learners in self-directed learning on the topic of early childhood education, specifically topics related to developmentally appropriate practices. Learners will develop individual learning plans, develop and research guiding questions for their learning, and connect with an instructor who will guide them self-directed learning process while they work through activities within 90 days. The goal of our program is not only to support knowledge gains, but also to support self-directed learning skill development.  Readers can view our original program design at https://selfdirectededac634.blogspot.com/p/program-design.html.  Our group reached out to professionals in a university setting and in a child care resource and referral setting to gain feedback on our program design. We were able to secure three evaluations on our program, giving us varied feedback to guide our revision ideas. 

Professionals’ Evaluation

Evaluator One:

Sarah Shepler

Department Chair

Business Administration, Supply Chain Management & Logistics, and Entrepreneurship

Ivy Tech Community College

What do you like most about our program design?

I appreciate that the program is somewhat self-paced with a 90-day deadline at the end.  I like the activities, and the progression of activities seem logical and to stackable within the end goal.

What do you think should be improved? Why? And how?

Does anything prevent the student from completing all of the work at once? For example, are there milestones that they should shoot for to ensure the activities are done over the course of the program and in order and not all at once or out of order (if that is the intent). From experience, having an assignment prerequisite is a great way to ensure students are completing items in order (if needed). So activity one must be completed before activity two is completed, and so on. This will also to help that students are not trying to complete all activities in the same day. It has been my experience that even the most self-guided, self-managed students need some boundaries and milestones to help them understand the best way to approach the program.

Evaluator Two:

K.P.

Program Chair for Early Childhood Education

Ivy Tech Community College

Roles:  Instructor, program administrator, create courses built upon statewide course objectives, student advisor, community partner, EC advocate,

Qualifications:  5 years teaching SPED Preschool; 10 years adjunct faculty for ECED; 5 years program chair for ECED; M.Ed in Curriculum and Instruction, EC emphasis.

What do you like most about our program design?

I like the idea of students creating their own timeline for due dates. I also like that students create their own guiding questions...learners will come into these courses with different experiences and levels of understanding on each of these topics. To allow them to decide what THEY want to learn prevents repetition and redundancy. However, having the input of the instructor will be helpful to make sure students are legitimately challenging themselves and growing.

What do you think should be improved? Why? And how?

There are a few typos, misspellings, grammar errors here and there (i.e.  "that will have some learner-led dates due dates")

Evaluator Three:

Michelle Kanable

Program Director

Bona Vista’s Child Care Solutions (Child Care Resource and Referral)

What do you like most about our program design?

In the work that we do with child care providers, I know that sometimes professional development can be difficult for individuals due to time and/or travel constraints. Offering a diverse method of PD is important. When offering a face-to-face training and incorporating a wrap-around technical assistance coaching visit, we can help gauge the understanding from the participant, as well as support the implementation on the knowledge gained. Face-to-face training blended with TA will always be my preferred method of training. However, when a distance learning or an online option is needed, you have come up with some unique ideas! As I read through the options that you have created, you offer several varieties that will meet individuals according to their skill set with technology, as well as their motivation to work at their own pace.

What do you think should be improved? Why? And how?

Here are a few things to think about as you review the programs you have proposed:

• IQ Project: If learners are expected to find and read scholarly articles on their topic, do you feel that the participant will know what resources are valid, or how to research a scholarly article? Many times, I wonder if participants even read the articles that we provide as additional research.
• Self-Directed learners: I feel that this may only fit a small population of the child care providers that we serve. Holding responsibility for creating their own timelines and working independently is a higher level skill of a student. If a participant has had negative school experiences, or hasn’t advanced in their own educational journey, this may be a struggle. This may be a better fit for high level thinkers. I wonder if this method would be a fit for a future Directors track. These participants typically have a higher education and have more experience in setting personal timelines. Directors may also have more experience in working with an online platform.

Overall, I appreciate the thought and planning that you have put into this project. I look forward to seeing how some of these activities might play out in possible implementation.   

Students’ Responses

Our group unanimously agreed with Evaluator 1 in the feedback that the order in which activities are completed is crucial. While our program does require that each activity is a prerequisite for the next, we felt that the program design could include more details about how it will work in the system that is housing the program. This will bring clarity about the progression of activities.   We decided that each activity would be a milestone toward the completion of the program. Since the program requires for learners to decide their due dates, they have ownership for meeting these milestones. To adjust the program, we will explicitly define the completion of milestones in the program design.

Evaluator 2 provided some really positive feedback that we were happy to read. Some of the major aspects of self-directed learning in the program design (learners creating their own timelines and guiding questions) were noticed as benefits of this program, which validated our hopes. The program is meant to benefit all of its intended learners because learners can develop their guiding questions based on their own interests and experiences, which builds motivation in each learner. The evaluator mentioned that it would be helpful for learners to get instructor feedback to ensure they are truly growing. One of our group members pointed out that when students submit their guiding questions to the instructor in Activity 2, it provides an opportunity for the instructor to offer feedback and push the learners to challenge themselves. As a group, we decided it was important to further detail the role of the instructor in the program design. As far as the typos go, we felt that this could be easily addressed with a closer edit from each group member.

One of Evaluator 3’s suggestions for improvement is related to the questionable expectation that learners in the program know how to identify scholarly sources. Our group agreed that the program design can and should include some support for this. Two of our group members specifically noted that the National Association for the Education of Young Children (NAEYC) could be listed as a resource for the program’s learners. Overwhelmingly, each of our group members addressed this evaluator’s second suggestion for improvement by expressing that the program absolutely takes into account the fact that self-directedness is not innate in every adult, and support is required and provided here. The self-paced nature of this program and support from the instructor each step of the way allows opportunities for each learner to grow and learn. One of our group members said that the instructor could be made available via phone if that would be helpful for a learner in this program. Another added that the learners can always reach out to the instructor if he/she needs any further clarifications or overall support - this could be made clear throughout the program design.

 Group Reflection

The most unique part of our program evaluation is the evaluator perspectives from which we obtained feedback. Our group interviewed two university professors, as well as a Program Director for a local Child Care Resource and Referral (CCR&R) agency. The higher education professionals gave us feedback that was helpful to consider from a higher-education perspective. The CCR&R Program Director was able to speak specifically on the clientele that this type of program would be designed for, and how the self-directed approach of our program would work with the typical person who the CCR&R system might interact with. It was useful for us to hear both types of perspectives to consider both how it might work out for the target clientele, who might not be used to a more self-directed approach, as well as the higher-education viewpoints which might tend to support more self-directed learners. Being able to obtain this feedback was significant to our group to consider how the design of our program might be able to be enhanced to best support our target audience in becoming more self-directed in their learning.

            Our group followed similar processes for the program evaluation as for our program design. Our group started with a conference call to discuss the project and review roles. Group leaders created a working schedule to help keep the group on track, and we made a few minor changes to the original plan of roles. From there, we individually set out to complete our portions then add to the group document for review and editing.

Table 1. Summary of Program Evaluation

	Evaluators	Ideas for Improving  program design	Revisions/Your responses
1	Evaluator 1	Plan milestones/certain activities for activity and completion. Consider pre-requisites or determining an order of completion of activities.	Specifically outline milestones as the learner determines them, thus leading to an order of completion for each individual learner.
2	Evaluator 2	Ensure instructor supports to learners. Stated typos and grammatical errors are present in current work.	Add more details of involvement to the instructor role: including having guided discussion with the learner at the time of research question submission and methods of contact with learner.   Review the program design for any necessary edits.
3	Evaluator 3	Work with learners to support their research skills and how to discern quality articles/resources. Plan supports for learners who are new to self-directed learning.	Identify some sources for learners to pull articles from for their activity. Identify how the instructor is available for ongoing contact and any additional support throughout the learners work.