Program Evaluation

I realize that the blog is not part of the Program Evaluation coursework this term.  But I have learned that it serves as a good outlet for me to 'talk through' some of the issues, without stressing about quotes and references.  (Yes, I know that using citations adds substance to one's writing, but right now I just need to get things straight in my head).

As has always been the case while enrolled in this program, time is such a huge factor.  With just less than two classes left, I'm ready to be done.  The University has not been kind to me either (they actually unenrolled me from the current course on Thursday of week 1 through a mistake on their part.  I didn't get things straightened out until Tuesday of Week 2.  Which means that I missed the discussion responses and assignments from Week 1.  On top of that, the professor is docking points off the W1 discussion grade because I didn't participate, AFTER the U told the professor that I was denied access through their mistake. Grr!  The professor did however grant an extension on the paper assignments.  And the U messed up the financial accounts, too.)

Anyway.

Even if I could put all of that aside, the course is difficult.  I am not fond of statistics or politics, both of which are part of the course.  Do I understand their inclusion in a course on evaluation?  Of course.  Do I recognize their importance and affect on evaluation results?  Absolutely.  Do I like dealing with either?  No.  Either subject can be skewed, twisted, and molded to be used for purposes on any side of an issue.  Both are nebulous creatures that creep up and are used for shock value (have you seen the political ads on TV?).

Ethically, morally, and for the sake of justice, evaluators should only convey factual information to stakeholders.  Evaluators should maintain distance from stakeholders, yet involve them in the evaluation process (leads to buy-in and action on program results). Evaluators should practice away from and above the political arena, lest they damage their credibility as objective assessors, past, present, and future.  (Fitzpatrick, Sanders, & Worthern, 2011; Schweigert, 2007, Mohan & Sullivan, 2006).

*See, I can't get away from the references, even when ranting!*

Truth be told, politics leaves a bad taste in my mouth.  I see enough of the currying favor behavior on my daily job as a school teacher.

I am hoping that my current outlook on the course changes (and quickly) but right now, my only thought is that I have 14 weeks to go.

References

Fitzpatrick, J., Sanders, J., Worthern, B. (2011). Program evaluation: alternative approaches and practical guidelines (4th ed.). Pearson, Boston: MA.

Mohan, R., & Sullivan, K. (2006). Managing the politics of evaluation to achieve impact. New Directions for Education, 112. 7-23, doi: 10.1002/ev.204

Schweigert, F. J. (2007). The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30(4), 394-399.

SOLD Improvements

I finally found a fix to the video hosting issue.  I loaded the video files to Vimeo (free service) and linked them into the Coursesites page.  It works great.  The only downside is that I'm limited on total file size uploads per week (500Mb), which translates to 2-3 video files.  It's not a huge issue, but may be a factor later on.

The test date of the project was moved up to this Friday (tomorrow)!  The admins want to run the program and make any final adjustments over the week.  If all goes well, the rest of the faculty will receive the email invite to Coursesites on Tuesday and the project will be live!

The first development module is completely done (that's the one to be tested on Friday).  A stand-alone video tutorial on fundraising is complete and just needs to be linked to the Coursesites page.  The Money Matters module is in process (one video is done, one yet to be recorded, plus the mod pages still need to be constructed).  I won't be able to add those videos until Wednesday of next week because of the way Vimeo limits uploads.

So far, it's going well.  The tech/video problem really added a lot of time to the project, but a workable solution was found.  Now on to the implementation and evaluation stages.

SOLD Project hits Scope Creep

After grinding out the PM course project, and a flying trip to PA and back for a family emergency, I'm feeling a bit wrung out.

But projects don't halt so that I can take a nap, so onward we go.... sort of.

I'm finding that communications, expectations, and tech issues are my biggest problems right now.  I was tossed the project (courtesy of my description of the Master's program) without much thought towards expectations, just a "do it" attitude.  The VP/SME that I am primarily working with for the first phase is not incredibly tech-savy, and does not take suggestions well that would make implementation easier.  I have been sending regular updates on progress, with a few questions for clarification.  He normally addresses the first issue, and ignores the rest (I limit my items to less than five).

Despite those issues, the majority of Phase 1 is done.  Learner analysis, tech analysis, and assessments are done.  I'm learning more about the tech issues as we progress.  I'm using the Coursesites LMS to host the development training.  One module has an .avi video, made with CamStudio, that lasts about 8 minutes.  The video will play, but there is no sound and some buffering/start/stop issues.  I've tried to compress the video to decrease the byte size, and have contacted the Coursesites tech support about the sound issue.  Since all of the video and file settings are correct, the tech crew thinks its a problem with their video service (QuickTime) and they're still working on it.

Until they figure that part out, I'm have to search for other options to host the video.  YouTube is blocked at our school.  The file size is too big for TeacherTube.  So I'm still searching.  As I'm operating on a $0-budget, the service would need to be free, have a 200 MB minimum or larger upload size, and not be blocked by our county's servers.

Before I could pinpoint the issues with the CamStudio video, the VP/SME went ahead and recorded two more screen capture videos for later modules.  I'm slightly frustrated.

1. He won't use a script - which leads to rambling and excess file size
2. I don't know the content of the subsequent videos or even if a video clip is the most appropriate way to present the content information

And the deadline is looming.  Everything is supposed to be live by next Tuesday.

Scope Creep: Historical Recreation

In my spare time, which there sadly isn't enough of, my husband and I are members of a historical reenactment group. The group focuses on historical recreations from Roman to Elizabethan eras.  Each person picks a particular era, location, and related activities in which to participate.  For example, my particular focus lies in Tudor-era France, stretching to Elizabethan.  Within that broad focus, I research and sew clothing that would be appropriate, along with all the accessories that would also be worn.  I also create bobbin lace, as would a lady during Elizabethan times.

Aside from some budget and time constraints (sewing things at the last minute), the scope creep could relatively be contained if I were the only stakeholder involved in the process.  Often, as with any volunteer group, the baton of leadership is passed from member to member to organize specific events or to coordinate activities for the whole of the group.  Such was the case with me about five years ago.  The organization typically holds the events over weekends outside.  The scope of the project would include security, some accommodations, activities, schedules, and an evening meal on Saturday night.

I was asked to organize a large event that would occur over 3 days in early November.  The regional heads of the organization had specific requirements.  My original bid for the event would safely land within budget and would meet most of the needs of the regional heads.  The original bid was accepted in early September. Then I was told that the original plan for the Saturday evening meal was not acceptable in late September.  This change in the dinner plan cause me to scramble for a new location for the event and readjust both the activities for the weekend, plus the event budget as the second venue was more expensive than the original.  The new location required their own security, plus payment for cleaning crews, and the dinner menu had to be changed (which caused the chefs who would have been preparing the food on site a large amount of displeasure as even the food and recipes are researched with to be historically correct, within safety guidelines).  Additionally, the regionally heads asked in October to change the schedule of activities for the weekend to include events that would require additional volunteers.  Other problems during the planning phase included the resignation of an assistant, a change in chefs, and volunteers changing positions and roles.

The majority of the issues during the project were related to the expansion the scope of the activities (Portny, Mantel, Meredith, Shafer, Sutton & Kramer, 2008).  Some were related to inexperienced people assigned to tasks in planning, namely myself.  Although the event turned out very well, several things would have improved the process.  It would have been hard to anticipate the some of the difficulty (all of the kitchen cabinets were locked when the chefs arrived), some could have been handled better, with a lot less panic on my part.  I did not get a written confirmation of the original bid, for starters.  As Vince Budrovich expounded on in the "Practitioner Voices: You Can't Win Them All" video this week, receiving confirmation in writing for original project goals and for any changes is crucial (Laureate Education, Inc., n.d.).  Secondly, the assistant who resigned was allocated the task of coordinating the building services of the kitchen and ceased to communicate in the middle of October with anyone (thus leading to the kitchen problems).  I should have been more involved with his contacts at the building for the event so that I could follow up on progress personally, rather than having relied on him.

I am glad that I took on the project, despite the problems.  I learned a great deal about the organization, and how to plan an event for the organization.  Sadly, though I have not had the time, opportunity, or the inclination to do so since.

References:

Laureate Education, Inc. (n.d.) Practitioner voices: You can't win them all. [Video]. Retrieved from https://class.waldenu.edu/webapps/portal/frameset.jsp?tab_tab_group_id=_2_1&url=%2Fwebapps%2Fblackboard%2Fexecute%2Flauncher%3Ftype%3DCourse%26id%3D_1342057_1%26url%3D

Portny, S. E., Mantel, S. J., Meredith, J. R., Shafer, S. M., Sutton, M. M., & Kramer, B. E. (2008). Project management: Planning, scheduling, and controlling projects. Hoboken, NJ: John Wiley & Sons, Inc.

Timely Studies Update/ SOLD Project

For those that were curious about the real-world project that I was asked to take last week, here is an update with more details.  The blog entries will be titled 'SOLD', so they are not confused with our grad class entries.

The planning and work has begun on the work-project that my principal asked me to undertake.  I've named it SOLD for Staff On Line Development.  I had an initial planning meeting with *some of the stakeholders last Thursday.  There were supposed to be 6-7 VPs involved in the project and only 3 were at the meeting.  One was not at work, two were conducting interviews for the upcoming school year, and one was filling for our head principal, who was also out of the office that day.  The meeting took longer than scheduled (1 hour versus the actual 2.5).

Although slightly underwhelmed by the VP participation, the meeting went well.  The missing VPs will likely be involved at later stages and when/if additional modules are added to the project.  We're using a series of web tools and resources to create and host the learning modules for the staff.  Since I had used the Coursesites.com LMS for a previous grad class project, my familiarity and the price-point of the site (FREE!), the modules will be hosted on Coursesites.  I've already started setting up the site and creating the content and module pages.

One of the VPs is taking an existing PowerPoint presentation, editing, and adding screen-video narration using CamStudios.org.  Another is using Adobe to create PDF forms.  The LMS will allow quizes and drop-boxes for assignments, ensuring that faculty members 1. complete the module training sessions and 2. allow for assessments and feedback to the project team.

The goal of the project is to convert some standard face-to-face content to the web-based forum.  Converting to the web-based forum will:

1. Improve pre-school time efficiency in preparing for the school year
2. Model technology practices that teachers may use with their own students
3. Deliver mandatory training materials to faculty members without the need for en masse time intensive meetings.

With decreased pre-school prep time and changes to teacher-contract hours, there is a growing need for the school to develop methods of delivering training and content to teachers without impinging greatly on teacher hours.  Previous methods of content delivery in face-to-face environments, either in whole-faculty meetings held after school or in smaller groups during teacher planning periods, have not been effective either in content retention or in meeting time restriction.  By moving the content delivery to a web-based platform, time restriction should be avoided by keeping delivery to smaller time intervals that can be completed over a two-week time period at the learner's convenience.

The school also places heavy emphasis on the use of technology in the classroom and in the school.  Previous attempts at incorporating technology applications have had the best of intentions with lack-luster results.  These results can be attributed to content-intensive learning sessions with little application or practice for teachers to effectively incorporate the technology into their curriculum and plans.  By modeling the use of technology through the Staff On Line Development project and through the content focus of specific modules, teachers will be able to refer to the learning module resources as needed, as well as have examples of the applications that they can conform to their specific needs.

The initial phase of the SOLD project will be to create a two-phase learning module on Substitute procedures (using the Sub Finder system and Creating Sub lesson plans/Emergency Lesson Plans) that will be launched by Aug. 21st, the date that the faculty returns to school from summer vacation.  Should the first phase/module meet expectations, additional modules will be added to include:

• School Money Matters (Field Trips, Fundraising, & Club Money)
• Classroom Management & Instruction (Tips & Tools for CM, Teacher Evaluations, Referral Systems)
• Student Support (Reporting Abuse/Neglect, Tier II Assistance, IEPs & 504s)
• School Website (Intro, Basic, and Advanced applications)
• School Safety (Emergency Drills/Procedures, County Required Safety Training)

Some of the modules will be required of all faculty members, some will be voluntary.  New teachers to the school (those with 0-3 years experience teaching at the school) will be required to complete more of the modules than veteran teachers.  Some veterans will be required to complete modules based on past-performance, at the VPs' discretion.  Assessments and feedback will be collected through a variety of applications, including quizzes, surveys, written responses, and form completion/drop-box submissions.

Key stake holders:

• Me - project manager/web developer/instructional designer
• Principal-client
• 6 VPs-instructional design team
• 1 Technology Resource Teacher
• 1 IT/Network specialist
• 250 faculty learners
• 2300 students/secondary clients
• 1 Financial officer (money matters)
• 1 Nurse (for county required blood-pathogen training)

Budget:

• Non-existant - I *might* be awarded a stipend for the work.  Otherwise, I'm paid in learning experience, resume' building, future administrative experiences, and real-world grad class applications.

Ask me questions, leave me comments.  The more I talk about the project, the greater the chance that I'll remember something important. Cheers!

Estimating Costs & Allocating Resources

I started this assignment by looking at the Microsoft Project 2010 website.  I like PC's, so Microsoft is normally the first place that I look for software and templates.  The software package seems to have everything a Project Manager could need in planning, tracking, and adjusting the budgetary and scheduling concerns of a project.  The only problem with Microsoft Project is that it is very cost-prohibitive for a lone Instructional Designer.  The site does offer some sound advice for preparing a project budget here.  Although advice is valuable, so are tools.  I started to look for more pocket-friendly varieties of software and downloads.

Bright Hub

Bright Hub has created several templates for Project Managers to use for free.  The templates are written in Microsoft Excel, which saves PMs from downloading or purchasing expensive software packages.  The download includes an overall budget, a monthly budget, and a task project budget page.  There are no instructions or tutorials to help complete the sheets, but the template designs are clear and easy to read.  I was very glad to come across this resource, as estimating and calculating cost for work-hours is a new skill that I am learning.  This template would save me some of the frustration of setting up digital tables, charts, and calculations.

Smartsheet

Smartsheet does have a small monthly cost for a single user, but it is much more reasonable.  There is also a 30-day free trial.  The web-based system limits the users to the number of sheets (templates/forms) and the size of the files (in Gigabytes).  The Smartsheets product includes many features that PMs would find useful, including Gantt charts, calendars, and collaboration between team members.  The product could help PMs with prioritization of tasks, budgets, performance reviews and task lists.

Timely Studies Part Duex

I talked with one of the VPs yesterday and found out a little more about what the administration wants in terms of the projects from my last post.  It is two separate roles, and I have an initial meeting with several stakeholders next Thursday to discuss the projects more.

Project 1:  Develop a single module for new teachers focusing on procedures and expectations for requesting a substitute in planned and emergency absences, as well as covering appropriate lesson plan content suitable to use with substitutes.  The module will be self-paced, housed either on the school server or loaded to each teacher computer desktop.  Completion would be assessed by a simple quiz or checklist, with the end result being a printout to be submitted to the administration by a certain day in mid September.  The module time frame is about four weeks (starting today).

If the project goes well, then additional modules would be added later.

Project 2:  I'm in charge of the staff development activities (SDA) for the year for integrating technology in the classroom.  The SDAs would occur once a month (totaling approx. 8 sessions) for an hour each.  I haven't been given any specific goals or areas to focus on yet, but my initial thoughts (based on informal conversations over the last few years with coworkers) have included the following:

1. Edline Basic: Setting up individual class webpages on the school website, inputting data on the calendar, uploading documents. (Edline is the website program the school uses).
2. Edline Advanced:  Using the Homework Hand In/Dropbox, creating online quizes, using the Discussion/Blog feature.
3. Wikis/Blogs/GoogleDocs for group projects and student collaboration
4. Podcasting & Videocasting
5. Flipped Classroom techniques for greater student engagement
6. Clickers for on-the-spot classroom assessment
7. Webquests/ Specific Content resources
8. Student Use of Web Content & Plagerism

If you have any other ideas for technology in the classroom, please let me know in the comments.