If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.
You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!
Without a doubt, the training program has been a resounding success. The students responded extremely positively to the training program, as evaluated by their informal feedback along the way, and in terms of their achieving the formal objectives: 90% of the student employees demonstrated increased confidence in their ability to support media studio patrons, and 86% of the students achieved a score of 8 or higher on the training rubric.
Objective Review:
Outcome
Objective Statement
Assessment Measurement
Increased ability to assess and support audio, video, web and graphic design needs for patrons of the media studio
90% of media studio student employees will demonstrate increased confidence in media needs assessment, and in Audacity, Illustrator, FCP and HTML knowledge.
Likert Scale
Basic knowledge about Audacity, Final Cut Pro, Illustrator, and HTML.
80% of media student employees will demonstrate proficiency in Audacity, Illustrator, Final Cut Pro, and HTML by scoring 8/12 or higher on the training assessment rubric.
Rubric
A quick recap of the training program is in order. The media studio is a do-it-yourself media production facility located in the library and open to the entire college campus. There are over two dozen student employees that provide support to the community on video, audio, web and graphic design projects. My training program was designed to address two problems: 1) the near impossibility of scheduling face to face essential training due to hyper-busy schedules, and 2) students tend to stick with what they know (e.g. audiophiles drilling deep into audio technologies and not ever learning the basics of video, web or graphic design). To address this, I developed a series of eLearning cross-training modules that could be performed asynchronously and independently by the student employees.
After much discussion with the students, I created four training modules, one for each of the areas that the media studio supports. They were Training Module 1: Sound Design Intro and Audacity, Training Module 2: Graphic Design Intro and Illustrator, Training Module 3: Video Tools and Final Cut Pro, and Training Module 4: Web Design Tools and GetWeb. Each module included a general overview of resources available in the studio and suggestions on what resources were best when patrons need help with their projects. The second and more dominant part of each module was technology skill training designed to address one common patron request from each of the four areas, namely Audacity, Illustrator, Final Cut Pro and GetWeb. The training modules relied heavily on video tutorials and lectures distributed through online video, both QuickTime and Flash.
Before the training begin, each student employee completed a pre-training survey. Then every week to two weeks, I assigned the student employees a training module to complete. Each module had a deliverable, which I tracked and scored. By the end of the four modules, I had point tallies for each of the employees. When the training was finished, they each completed a similar post-training survey.
As the students sent me the deliverables (via email), they sometimes provided comments. I wrote down interesting things they shared in conversation. The surveys also contained a field to provide feedback. The aggregate log for their comments is found on the Student Employee Feedback page. The high completion rate and positive feedback quickly indicated that the students enjoyed the training and thought well of the program. A few representative comments can be found below:
"It was good to have a focused assignment. It's only when you need to make something that you actually learn it." - Andrea '08
"I really appreciated the training modules. They weren't too in depth, but I now have the basic skills in all of our programs to at least direct patrons in the right direction. Definitely a good addition to cloisters staff training." - Marshall '12
"These videos are a great way to teach people. That guy (Cristian) is a great speaker." - Nick '11
"This poster is certainly not the best poster ever made but I know much more about how to do things in illustrator now and have a better sense of how to help people. "- Kelsey '12
"You are so good at making those videos." - Madison '11
The student employs exuded endearing enthusiasm. I created a page hosting most of the student work. One can view examples of the sound pieces, posters, videos and web pages they created. The final training module had them host links to their other deliverables in a web page. Here are a few examples:
Web Page Example - This is the first web page this student had ever coded.
Audacity Example - This is the first time this student had used Audacity.
Poster Example (thumbnail) - First time using Illustrator
Brian substituted this work for his FCP training piece. He is an experience filmmaker. I allowed to substitute prior work as needed; I didn't want to assign training "busy work".
The goals of the training program were achieved both in terms of the feel of the program and in terms of achieving the target objectives. It was a very successful pilot project has become a cornerstone of our student employee training programming.
Although evaluating the main objectives seemed fairly straightforward, a more complete picture is drawn by further analyzing the data. The pre-training raw data and the post-training raw data were collected using the live form feature from a Google Docs spreadsheet. When reviewing the data and comparing it to the objective of "90% of media studio student employees will demonstrate increased confidence in media needs assessment, and in Audacity, Illustrator, FCP and HTML knowledge," I realized that the summary of all responses was less important than each student employee demonstrating increased confidence. I defined "increased confidence" as a more confident answer to at least two of the ten questions. I was fortunate in that all twenty-nine of the student employees had completed both surveys. I compared each student response in the pre-training survey data with the corresponding response in the post training survey data. As defined, 26 of 29 of the student employees demonstrated increased confidence. This equaled 90% exactly, which was targeted goal. The three students who didn't indicate increased confidence were already very expert (two students) or weren't able to complete the bulk of the training (one student). Hitting the mark exactly is a reminder to educators that we should set our aims high in order to help students achieve their best.
In terms of analyzing the rubric, it was straightforward. Every student that completed a training module and submitted the deliverable scored full points on the training. There were no cases of students completing the training but not being able to submit a satisfactory deliverable. The training modules worked as intended; the program was successful as designed. The data also indicated the great advantage of pursuing a cross-training approach. Maximum benefit from training was derived when those performing the training were less knowledgeable.
Below, I have examined each survey question (Q1 - Q10) and compared the pre-training data with the post-training data and described the numbers that stood out as significant.
Overall there was a strong shift towards increased confidence in providing help to media studio patrons.
Student employee confidence in their ability to assess patron needs increased, especially in the "strongly agree" category.
29 of 29 students completed the Sound Design Intro and Audacity module. Q3 demonstrated a very successful module, as the 27% of employees that expressed a lack of confidence in their ability to provide help with sound projects became confident in their ability to help others. The end result was 94% feeling able to assess sound needs, (compared with 62% at the onset).
The Audacity training was incredibly effective. All student employees completed this module and the results reflect their efforts as 96% self-reported Audacity core skill mastery, compared with 52% before training.
27 of 29 students completed the Graphic Design Intro and Illustrator module. The numbers here are less striking than in other modules. On the plus side, those feeling unable to help assess graphic design needs went away completely. However, those feeling confident increased only slightly. This may have something to do with the complex nature of patrons' graphic design requests; they tend to be fairly unique and challenging.
The Illustrator core skills training effectiveness was reflected in a huge jump of student employees feeling "strongly confident" in their Illustrator skills, from 28% to 58%.
24 of 29 students completed Training Module 3: Video Tools and Final Cut Pro. There was a huge jump in student employee confidence in their ability to provide help to patrons with video needs.
Before the training began, 41% had no confidence in their ability to perform basic edits in Final Cut Pro. This dwindled to only 3% undecided, with a full 92% expressing confidence or strong confidence in their ability to implement the core functions of Final Cut Pro.
The Training Module 4: Web Design Tools and GetWeb training module had the weakest completion rate; 18 of 29 (or 62%) of student employees completed it. The last two modules competed with Thanksgiving break and the end of the year onslaught that characterizes the end of term. Still, there were significant gains in student employee confidence, but results were not as demonstrable as they were in the other modules.
Despite the lower percentage completion rate for this training, those that did complete it demonstrated significant gains in their confidence levels.
The student employees expressed a strong consensus that the training program was a valuable use of their time.
The first two modules enjoyed a nearly 100% completion rate. Modules III & IV did not fare so well (83% and 62% respectively). My training module production was delayed for the graphic design module, as I relied on a subject matter expert (a colleague of mine in Academic Computing Services). The module was ultimately better than if I had done it myself, but it hurt the timeline of the entire program. As such, modules III & IV competed with Thanksgiving break and then the end of term. Tom summed it up well, "You'll notice I've disagreed with the comments about web design confidence... that's probably due in part to the fact that I haven't gotten around to the fourth tutorial yet! Sorry! I've been writing huge final papers and will do it within the next couple days, promise." The converse of this was expressed by Andrea, "Doing this at the beginning of the year would be better timing, and help new staff get started." It would have been much more effective to have had two full weeks of school during for each and every one of the modules. Modules should be timed to not compete with either vacation breaks or peak work times of the academic year.
Now that I have a grasp of the overall project shape, I can do a better job next time of having the student employees take ownership of the training modules. As they are already organized into teams, each team will help target the needed learning for all employees. For example, the graphic design team would be in charge of developing and supporting the training for a Photoshop tutorial, should that be determined to be where the subject of the next training module. If they aren't able to actually create the training materials, they could play a more active role in promoting and supporting peer learning. I received many suggestions about what to do next, a few examples:
"Adding ones for photoshop and garageband might be a good next move. / Thanks again," - Nick '11
"Awesome job! Next step = advanced training modules. Also, module on SCANNING and best settings/way to save." - Brittany '09
Most of the students seemed to understand the necessity of doing the training asynchronously. But the fact that they moved through the training materials en masse certainly contributed to a sense of solidarity. Weaving in optional workshop times for them to do the training in small groups would have increased participation and opportunities for collaborative effort.
There were a few technical hiccups that hampered the training modules. Next time I will double check the installations of Audacity, as a few of the machines did not have the library file needed to export the .mp3 file, and this caused some consternation and delays. Specific training in Vspace would have saved some trouble in the beginning. I assumed that everyone knew how to access this campus-wide file repository but my assumption was incorrect. This was, however, an unintended positive result of the training in that everyone learned how to share files in Vspace and to share them on the Internet and the Intranet. Additionally, troubleshooting is an essential part of the experience in our studio and a core experience that has bonded the students many times over.
In terms of terms of methodology, one could critique the module completion rubric or the validity of using a Likert scale measuring confidence levels as sufficient proof of concept. In my research, I found very scant evidence of objective assessment of student employee training programs. Standardized testing is not ubiquitous like it is in K-12 environments. Usually the formal papers written about training programs were developed to describe training programs put in place after a situation had degraded to the point where intervention was absolutely necessary. Compared to the evaluation practices published, the media studio training evaluations compared very favorably. The pass/fail model of licensing procedures (teaching certificates, Final Cut Pro certification, the bar exam etc.) is a precedent for evaluating technical training. One is either certified, or one isn't. When it comes to helping other people, a high confidence level is critical. Many of the patron requests that student employees in the media studio receive are unique. A high confidence level empowers them to use what they've learned in the past to apply to the new situation. A superior method of evaluating results could be pursued, especially if the scale of the training were increased. The methods I used were low-pressure and non-threatening, which was a plus. The cross-training eLearning modules were very effective in providing training and boosting confidence, especially in areas where the students had little or no experience.
There are several avenues possible for sharing the best practices that emerged from this training program. The very least would be to offer a presentation for other student employee managers in Computer Information Services on our campus. This would include Media Resources, the Help Desk, and User Services. A broader application of these principles would include the college libraries and the learning and teaching center. The broadest possible application would be to include all student employee managers on the campus, though I think this would be overkill. Information technology and information services student employees would garner max benefit from an asynchronous eLearning training program, supported by face to face interactions.
Gada and Wagner (2000) point out the advantages of centralizing student employee training programs. At the University of Rochester, they revaluated the student staff structure and reengineered the process to be "efficient on a larger scale." Students hired by one part of information services could transfer to a different IT section with relative ease, while staying under one umbrella as far as training and achievement tracking. Our information services department hires and trains (or fails to train) in silos. Centralizing services at a small liberal arts college may not be a good match in terms of institutional culture, but as there are plans on the books to develop a new technology center at the center of campus, having a more strategic and systematic way to train student employees might likely prove useful.
There is a high school in the area that uses students (as part of classwork) as a sort of help desk. Perhaps this model might appeal to them (and other local schools) as a way to standardize knowledge and training. Perhaps what they are doing is already more advanced.
Many of the research materials I found beneficial were from professional higher ed oriented organizations, such as NERCOMP. I feel a certain obligation to share my findings with others I met during my participation in NERCOMP training events as well as those I keep in contact with via listservs and other social networking tools. Perhaps too I could publish a paper based on my findings.
The video tutorials and asynchronous eLearning materials were incredibly effective, especially in the context of the personalized learning of the media studio student employees. The students came to work there because they were interested in learning new things. They had a voice in what we learned. They could get help from me or from each other in order to complete the training. When I interviewed and hired the students, I asked the students to demonstrate something using technology. I primarily looked for the ability to be articulate and a desire to learn. The modules provided a structure and an opportunity for them to capitalize on the opportunities that attracted them to the media studio in the first place.
Having eLearning materials in the bank has proven useful to me on a number of occasions. Two administrators expressed an interest in learning Final Cut Pro. I pointed them at Training Module 3: Video Tools and Final Cut Pro. They were very appreciative. Towards the end of the semester, a half dozen students came in with poster projects. I was able to support them by pointing them at Training Module 2: Graphic Design Intro and Illustrator. I was still available to answer questions as they came up. But there's no way I could have sat down with each of those students as they came in and given them a one-on-one ninety minute tutorial on graphic design and poster creation. I have sent the links to faculty to post to their Blackboard sites in as I knew they were engaged in projects using video, sound or posters. The eLearning materials were a sort of "force-multiplier" for education, as once they were developed, I have been able to re-purpose them as needed. The training modules provided many opportunities for confluence of learning.
My job description prioritizes instructional technology support for faculty and students. But community members come to the media studio from all over the campus. The media studio is a place people come to for help with multimedia. Our campus has a help desk and a trainer. Often the lines are blurred between community members would seek out one place or another. I have sat in on meetings where a half dozen IT professionals debated why someone would go to one resource or another. If the professionals have this much trouble figuring it out, what about the average user? Perhaps the maximum advantage could be achieved by sitting down with user services and creating a list of the most often requested help items and creating eLearning materials for those.
Vimeo tracks total views and views by date.
With the ease of use of social networking tools and free online video distribution, it becomes relatively easy to track if students are accessing materials. I preferred the use of Vimeo to just the quicktime objects (for the Audacity materials) for this reason. When the dust settles on this project, I will retool all of these materials for publication to the media studio web site, and I will also put the Audacity training in Vimeo. The students were very comfortable with the video tutorials. But future plans should include video that includes closed captioning.
The maximum benefit of the training program was achieved for the "trainees" (those hired at the beginning of this semester). This may seem fairly obvious, but the trainees had the most to learn from training, and also seemed to enjoy it the most. Marshall '12 wrote, "I really appreciated the training modules. They weren't too in depth, but I now have the basic skills in all of our programs to at least direct patrons in the right direction. Definitely a good addition to cloisters staff training." Several trainees expressed little or no confidence in several areas. By the end of the program, they were confident. On an anecdotal level, last year, my first year as the manager, when there was very little structured training, we lost four or five employees. This semester we had a 100% retention rate. The training program provided a rich context for supervisor-employee interaction, a communication model superior to the normal dialogue of payroll and scheduling.
As we look to the future of our small liberal arts college, Gada & Wagner (2000) provided a model for centralizing and warehousing training materials for student employees in IT. The vision of students being trained and then being able to move from Help Desk to Media Resources is an appealing one. Manly (2000) points out that training is an iterative process, "The student employee training program must adapt as technologies emerge, customer expectations grow, and organizational foci change. Success requires an on-going review and revision schedule." Right now our college is exploring various online video services but we lack a central strategy. We are innovating in silos, but how do community members know where to go to find things? This is something we'll need to address, both in terms of warehousing out training materials and providing academic technology support to the community.
Both the students and my research indicated that that several tiers of training are appropriate. There is the initial training that new hires need and then there is the need for more advanced training. Manly (2000) provided "two levels of training, the first appropriate for entry level employees with limited background inthe specific technical area. A higher level course was also planned for the returning students." Kelsey '12 made some great suggestions, "A tutorial on troubleshooting all these programs would be helpful. Also, buddying trainees with more experienced workers at the beginning of the year for training time would be a good idea." Training earlier is better.
The asynchronous video tutorial based training module has become a cornerstone of the program. Its success though depended on the rich human relationships in the media studio. Student involvement in the planning and execution of the training was an essential part of the process. It would have been impossible to schedule four workshops for each and every one of the student employees. The eLearning model provided more opportunities for structured training than face to face training would have. The face to face training was there in support of the eLearning. The students often completed the training in pairs or small groups. The eLearning video training model was a good match for the 21st century learners in the media studio.
Gada, J. and Wagner, J. (2000). Changing the face of student employment in IT at the University of Rochester. In Proceedings of the 28th Annual ACM SIGUCCS Conference on User Services: Building the Future (Richmond, Virginia, United States, October 29 - November 01, 2000). Retrieved November 15, 2008 from ACM Digital Library.
Manly, C. M. (2000). Kicking it up a notch: adding zest to your student employee training. In Proceedings of the 28th Annual ACM SIGUCCS Conference on User Services: Building the Future (Richmond, Virginia, United States, October 29 - November 01, 2000). Retrieved November 15, 2008 from ACM Digital Library.
Comments (0)
You don't have permission to comment on this page.