cms @ ec >> Evaluation / Features >> Moodle

Introduction

Although a survey of faculty opinions about Moodle had already been taken in the Fall of 2004 the response was rather low and research on the Moodle system in the spring of 05 revealed a number of interesting questions that could be addressed. The decision was made to use a different online survey system, this time a javascript based system at SurveyMonkey.com. Although this allows only 10 questions to be asked (in the version available by free subscription) the advantages of the survey software (mulitple answers possible to questions, recording comments as well as choices, scaffolding questions with introductory text, javascript interface) were such that this was deemed to be an acceptable compromise.

All faculty using Moodle are enrolled in the Moodle course Moodle Workshop and so the News/Announcments activity was used to send email messages to all these faculty inviting them to complete the online survey. The initial message was sent on 19th April to the 60 faculty enrolled in the workshop, a reminder was sent on April 21st and a final plea for participation was sent on April 25th. Of the 60 faculty canvassed 32 responded, a participation rate of better than 50%.

go to top of page

Questions asked

Following the pilot trial of Moodle in the spring of 04 with two classes the system was made available to faculty in the fall of 05 on a local server but with little fanfare or publicity. However, there was massive interest from faculty and over the span of two semesters 100 courses were created. Of these 100 courses 21 remained unused, with a similar proportion for fall and spring semesters, [link] and the first question was designed to address this phenomenon. I had heard from a few faculty who hadn't used their Moodle course that they had got started too late and I figured that the time taken to learn the Moodle system might be an issue, hence the first question was addressed to those faculty who had asked for a course to be created but not used it. In the fall semester 27 faculty used Moodle with at least one course and the following spring 33 used it. However, these figures concealed a considerable turnover for no fewer than 23 new faculty started Moodle afresh in the spring and only 10 faculty continued using it from the fall semester with 17 dropping out [link]. Question 2 was addressed to those faculty who had used Moodle in fall but had decided against using it again in the spring (17) to try to ascertain the reasons for this. Conversly, since a total of 50 faculty had at least dabbled with Moodle over the two semesters, there must have been some reasons that attracted them to use the system which was the subject of question3. Given that time spent learning the Moodle system would directly impinge upon course preparation time I wanted to discover how much time faculty felt they spent in getting their course established on Moodle (question 4). And was it worth the effort? Did using Moodle help course management and/or teaching effectiveness (pedagogy) (question 5)? Subsequent questions dealt with support for Moodle, and other digital technologies. Question 6 enquired about types of Moodle activities faculty would be keen to learn about, and question 7 concerned workshop topics about digital techniques allied to Moodle that faculty might be interested in. It's often difficult to assess what sort of assistance with Moodle faculty work best with and when to deliver that. These issues were addressed by Questions 8 and 9. Finally, I wanted to discover how many faculty were planning to use Moodle next year (a good measure of how successful the system has been).

go to top of page

Survey Results

1. Unused Courses
Please comment on the reasons that you did not use the Moodle course that was created for you (select all applicable and add comments if desired).
 Response PercentResponse Total
 The course was set up too late in the semester
 0%0
 It seemed to take too much effort to start up with Moodle
 0%0
 I just didn't have time to learn it
66.7% 2
 I didn't have time to scan material for use as resources
50%1
 Any other reason or other comment
 0%0
Total Respondents   3
(skipped this question)   29
2. Change in which Faculty used Moodle

Faculty with Moodle course in the Fall but not the Spring

What reasons did you have for not using Moodle in the Spring semester? (write comments in the 'other' box if you want)
 Response PercentResponse Total
 My spring courses were not suitable for use with Moodle
50% 3
 I spent too much time messing about with Moodle in the Fall to bother with it again
 0%0
 I just needed to take a break from Moodle
16.7% 1
 It didn't really do what I wanted it to do
 0%0
 I spent too much time scanning material for resources in the Fall semester
 0%0
  Other comments (what made Moodle difficult?)
50% 3
1. Our course was only offered in the fall
2. I worked on the Moodle course through Bethany.
Total Respondents   6
(skipped this question)   26
3. Attractions to Moodle
What attracted you to using Moodle for your course in the first place ? (Select all applicable)
 Response PercentResponse Total
 I was persuaded or influenced by my colleagues that Moodle was a Good Thing
43.8% 14
 I understood that it was easy to get a course up and running
56.2% 18
 I wanted to make class materials available for review purposes
68.8% 22
 It was the only way to present copyrighted materials to the class
37.8% 12
 I wanted to have students submit their work online (eg assignments)
37.8% 12
 I wanted to have students do some writing online (discussion forums, journals, etc)
18.8% 6
 I just wanted to try out online course management
50% 16
  Other (please specify)
37.5% 12
1. We had to do that course online and Moodle was the format
2. desire to cut down on per-student costs of weekly copying of materials
3. I wanted to make stuff available to someone off campus.
4. It was there.
5. It was a great tool for viwing works of art...a good complement to my scanning project in the library.
6. I wanted them to have access to scholars at other institutions.
7. This seemed like a mostly efficient way to get course readings on "reserve" w/out the hassle of printing.
8. It was a great way to communicate with students about class materials and upcoming events.
9. I realized too late that I needed to use several copyrighted articles that I had not covered in course fees. Moodle rescued me!
10. Initially, I used it in a F/K as a clearinghouse for the work of the whole group. INDISPENSABLE! In spring, I used it in 2 other courses as a quick way to get out copyrighted materials, link on-line resources, and send messages.
11. Used CHEF and needed to try MOODLE
12. It was a good way to spread the class photo.
Total Respondents   32
(skipped this question)  0
4. Time Spent Learning Moodle
How much time or effort did you spend getting to grips with Moodle before and during your course (check nearest answer):
 Response PercentResponse Total
 It seemed like very little time (a couple of hours)
43.8% 14
 Not too much but noticeable
31.2% 10
 Quite a bit of time but reasonable
21.9% 7
 I had to spend too much time
3.1% 1
 I had to spend an inordinate amount of time and effort
 0%0
  Comments on your choice
25%8
1. the whole put in and move around takes time. But I expect to do it better in the fall because I'll start getting it together in the summer.
2. Not really applicable with me, because I missed the course for faculty and had my own tutorial with Marc. It was a bit difficult though, without a lot of computer skills.
3. I spent quite a bit of time- time taken away from my course and students- but not enough to really understand or use Moodle effectively.
4. Even so there were things I still did not get the hang of.
5. IT was easy, With Mark Pearson's very generous 1 to 1 instruction.
6. I also had Mark do a session w/ students in my F/K in the fall so that in the spring, rather than put up articles and links myself, I turned all the work over to a trained work study student who knew what to do from the F/K class. EASY!
7. Moodle seems pretty easy to use (at least the parts I have used so far). The most time is simply uploading files, and scanning (!) all the materials in. It really was faster just to drag and drop to a groups folder. However, I like the fact I can organize the files in a more logical manner.
8. I didn't do as much as I planned to do because I didn't have all the time to try things out and do various things. I could get some aspects done. Also, I don't have a scanner, so I had to use my secretary for that, which was mostly fine.
Total Respondents   32
(skipped this question)  0
5. Moodle as Course Management
Having used Moodle in at least one course, was it worth the effort? (add comments in the 'other' box if desired)
 Response PercentResponse Total
 I think that my course was better from having used Moodle - it made managing easy & enhanced pedagogy too
34.4% 11
 Moodle made managing the course easier but had little effect on pedagogy
53.1% 17
 I discerned no difference between using and not using Moodle
 0%0
 Using Moodle made a difference but it was too much effort
6.2% 2
 Using Moodle made no difference to the course and it was a waste of time
 0%0
  Other (please specify)
37.5% 12
1. I'm not sure I'd say easier, but I can imagine the management of material will improve.
2. Moodle muddled my course. It lead to unnecessary confusion. The suggestion that students would understand and be able to work with it naturally did not prove to be true.
3. I love moodle!
4. This was a fast way to get good images for PPTs and make them immediately available to students. I discovered, however, that more than a few students relied on the available moodle ppts rather than my class. When students make paper copies of the ppts before class and then attended class it seemed to provide them more time to think about the material presented, rather than spending time copying images, lists, etc.
5. I got lazy sometimes and forgot to post stuff I had promised students I would.
6. Moodle is a great resource to have available. We badly need the flexibility it gives us.
7. Students reported frequent internal errors with trying to submit online assignments.
8. I used Moodle in all 6 of my courses this fall and spring; I used it heavily in one course per semester and lightly (in two cases, almost not at all) in the rest. For me, the larger the class the more use I found for Moodle; somehow the smaller classes also tended to be more contained within the classroom and not need Moodle as much.
9. Useful providing direct links for the on-line text.
10. I really got started too late (right before the semester) with moodle to take advantage of some of the features that would influence pedagogy more. mostly, it just made distributing copies of lectures and resources easier.
11. I primarily used Moodle for getting information and course materials to all students. I very much want to explore adding the grading and quizzing feature SOON!
12. I only used it mostly to post homework asssignments on the web, however this had a small but noticeable affect on how smoothly the course ran.
Total Respondents   32
(skipped this question)  0
6. Taking Moodle further
What Moodle 'activities' might you be interested in using given the right help:
ultra keenkeennot keenN/AResponse Average
News [Make course announcements]
32% (8) 40% (10) 20% (5) 8% (2) 1.87
Assignments [Writing or other assignment activities - offline or file upload]
22% (6) 48% (13) 15% (4) 15% (4) 1.91
Forum [Discussion forum - group, whole class, single or multiple topics]
21% (5) 29% (7) 33% (8) 17% (4) 2.15
Journal [Online writing assignment with active web links]
20% (5) 24% (6) 36% (9) 20% (5) 2.20
Blogs [Online writing in weblog type format lasting the whole course]
4% (1) 24% (6) 52% (13) 20% (5) 2.60
Choice [Students presented with a question and up to 6 possible responses]
4% (1) 21% (5) 46% (11) 29% (7) 2.59
Groups [create, allocate students, apply to activities]
12% (3) 24% (6) 44% (11) 20% (5) 2.40
Quiz [Multiple choice. Can be auto-graded or used for review. Can include images]
12% (3) 28% (7) 48% (12) 12% (3) 2.41
Pedagogical tools [Workshop - peer assessed assignments, Glossary]
23% (6) 50% (13) 15% (4) 12% (3) 1.91
Total Respondents   29
(skipped this question)  3
7. Going digital with your class
Please indicate any of the areas below that you might be interested in following up with (and do make other suggestions too)
 Response PercentResponse Total
 Transferring classroom or paper procedures into digital form (Moodle or other)
48.4% 15
 Sharing Moodle techniques / insights / best practices with other faculty
64.5% 20
 Using Moodle discussion forums to extend discussion outside of class time
38.7% 12
 Digital imaging - processing digital camera images in photoshop, scanning for powerpoint, video making and streaming
45.2% 14
  Other suggestions
12.9% 4
1. Best ways to use group or collaboration online
2. How do I bring in scholars from other institutions without them having to learn how to negotiate the Moodle interface?
3. Not sure. Also, I was not sure about the previous item's questions.
4. It would be super if every entering student who was in the facebook could be entered into the system with their picture. That way we wouldn't have to deal with taking and uploading pictures for them.
Total Respondents   31
(skipped this question)  1
8. Assistance with Moodle - Format
Choose any of the options below and make a suggestion if desired:
 Response PercentResponse Total
 One on one help (Instructional Technologist & Faculty)
67.7% 21
 Online documentation (please suggest areas to cover and depth)
22.6% 7
 Printed manual (ditto)
25.8% 8
 In house workshop lead by Instructional Technologist (similar to the New Year's workshops)
38.7% 12
 Workshops lead by outside expert
  3.2% 1
 Peer exchange with other teaching faculty (as in the Moodle Moot in the Fall)
48.4% 15
 Meet with faculty from other colleges under the auspices of MITC (Midwest Instructional Technology Centre)
6.5% 2
  other suggestions
22.6% 7
1. Some mentor relationship where there is some expert available to answer questions quickly as needed.
2. all of it
3. ONe on one instruction was very helpful to me; but my case may not have been typical because I just wanted Moodle to present some readings.
4. Online documentation, i.e., how to upload a document or put in a link.
5. The ways faculty use Moodle seem quite diverse, or it seems so to me. I'd welcome specialized workshops for teachers in different areas, and advice and ideas for utilizing Moodle for particular goals (for me, ideas about using Moodle for literature and film classes).
6. All are workable and have merit (Wes)
7. I like one-on-one help, but I realize I was offered it and didn't use it, because I let such opportunities slip away. --Kari
Total Respondents   31
(skipped this question)   1
9. Assistance with Moodle - Timing
Select any of the options given below or add your own.
 Response PercentResponse Total
 Short sessions (1 - 1 1/2 hrs) during the semester
64.5% 20
 Regular biweekly sessions during the semester
6.5% 2
 Whole day session just before the start of the semester
22.6% 7
 Whole day session during the summer (including May)
32.3% 10
 Weekend session
 0%0
  Other (please specify)
22.6% 7
1. It juswt depends!
2. who knows?
3. The immediate access answer to the last question.
4. not sure
5. Probably good idea to front load or offer during the beginning of the semester. My guess is attendance would drop off by the end. (Moodle users bash at the end of the term to debrief?)
6. Short sessions during the semester with TOPICS announced...for continuing users. Whole day session for new users (and review for others) at start of semester.
7. only one day, though--two-day sessions are too long for me.
Total Respondents   31
(skipped this question)   1
10. Planning to use Moodle next year
Please indicate in which semester you might use Moodle next year (05-06)
 Response PercentResponse Total
  Fall only
10.7% 3
  Spring only
17.9% 5
  Fall and Spring
60.7% 17
  Neither - I don't plan to use Moodle
10.7% 3
Total Respondents   28
(skipped this question)  4
go to top of page

Discussion

As mentioned above 20% of the courses created were not used; I wanted to find out whether there was anything I could do about this. The few responses to question 1 (3 from 32) focussed on the lack of time for faculty to learn Moodle or to scan resources rather than on their motivation or lateness in setting up the course. Discussion about the amount of time taken to get to grips with the Moodle system is continued in the context of questions 4 & 5.

I was also concerned about reasons faculty may have had for dropping their use of Moodle after the Fall semester. Although there were only 4 responses from 32 to question 2 a clear pattern emerged. It seemed that the reasons for dropping the use of Moodle were indeed neutral ones — either the Spring courses were judged unsuitable for using with Moodle or the course was only offered in the Fall or the teacher just wanted a break from using Moodle. On the other hand there were no responses to options such as Moodle not doing what the teacher wanted it to do or spending too much time and feeling that it was not worth it. Thus it seems that despite the large turnover in faculty between the Fall and Spring semester there was not widespread discontent with the system and that the genesis of this phenomenon was indeed neutral in origin.

Question 3 (attractions to Moodle) was answered by all respondents who were able to choose multiple answers. The most popular response (68%) was wanting to make class materials (usually Powerpoint lectures) available for review for an exam or a paper assignment. Faculty were also attracted by the perception that Moodle was easy to get going with (56%) — and they did not seem to be disappointed, see responses to question 4. Next in popularity (50%) was the response that faculty just wanted to try out online course management, and this may be part of the key to the 'turnover' phenomenon — having tried Moodle in the Fall some faculty may not have felt sufficiently motivated to use it again in the Spring semester. Colleagues played a role in influencing each other to use Moodle (44%) and some of the comments such as “Moodle rescued me” and “INDISPENSIBLE” indicated a high level of enthusiasm. However more prosaic reasons were common; the fact that Moodle was password protected meant that it was the way that faculty could present copyrighted materials online (38%) and also have studnets submit their work online (38%) or do online writing (19%). Finally, faculty found new communication possibilities with Moodle for upcoming events.

All the respondents also answered question 4 (time spent learning Moodle) where they were asked to check the single nearest answer. The majority of respondents (44%) considered that it took little time to get to grips with Moodle for their course. However, 31% felt that the time they spent was noticeable and a further 22% considered that they spent a sizeable amount of time. But only a single respondent felt that he/she spent too much time learning Moodle and there were no faculty for whom the process of learning Moodle took an inordinate amount of time. The comments were revealing. My hunch is that faculty who are comfortable and familiar with computers and especially online applications (such as squirrel mail or webdb) found Moodle easy to pick up and run with, but the Moodle interface does require a set of computer skills which may be underdeveloped in some faculty who want to use it — hence the longer time spent getting to grips with the system. Some faculty still felt that they were still not using Moodle effectively or that they still had not really got the hang of it by the end of the year (my frustration is that of identifying these faculty so that I can follow up — make yourselves known to me, chaps!). There were also workflow issues with scanning material — we do need a creative solution to the issue of scanning lrage amounts of material for online resources. Finally, one respondent mentioned the convenience of 'drag and drop' compared to single file upload that Moodle employs. Here is where integrating the Webdav system into Moodle would be a great boon since this would enable drag and drop for multiple files.

The logical corollary to questions about time spent learning Moodle is was it worth the effort, which was the subject of question 5. Most faculty (53%) felt that Moodle was beneficial in terms of managing their course but that it did not have much effect on pedagogy. However a sizeable proportion of respondents (34%) felt that their use of Moodle did enhance their teaching as well as assist in their management of the course. Only 2 respondents felt that their use of Moodle involved too much effort even though it made a difference to their course. And there were no faculty for whom Moodle had no effect whatsoever or who felt that it was a complete waste of time. In the comments section most faculty reported positively on their use of Moodle. One person found more use for Moodle in larger classes, another commented that it had a small but noticeable effect on the course, and another wrote that students bringing copies of powerpoint presentations into class gave them more time to think about the material. However, one faculty found that Moodle muddled the course and reported that the students had difficulty with it, and another said that students had reported errors with submitting online assignments.

Not surprisingly, most faculty made limited use of the features available in Moodle [link] and so I wanted to find out which activities they might be interested in pursuing further (question 6). Faculty seemed rather keen to discover more about making course announcements (News), and online assignments and there was a measure of interest in discussion forums. Surprisingly, there was a good deal of interest in explicit pedagogical tools such as the workshop. Even though there was less interest overall in the online writing activities such as journal or blogs, nevertheless a substantial minority showed interest. Similarly for choice, groups and quiz activities there was interest expressed.

What other digital techniques are of interest to faculty? Of the options presented in question 7 (going digital with your class) sharing Moodle techniques and best practices with other faculty was the most popular. Next was sharing ways in which faculty had moved from paper procedures to digital to save time & effort, and digital imaging involving scanning, photoshop & powerpoint was also of interest. Faculty also asked about getting student mugshots straight into Moodle and how to have scholars from other institutions access the course.

What is the most popular format for Moodle assistance and when should this be delivered? These were the subjects of questions 8 and 9. One on one help was the most popular with peer exchange of ideas and best practices second. In house workshops were of interest but online or printed documentation was generally of less interest. There was very little or no interest in meeting with other Moodle using faculty or getting outside experts in Moodle. A comment referred to a need for expert help available locally so that quick answers could be forthcoming. Teaching strategies for using Moodle in different subject areas was also requested. In terms of timing, short sessions sessions during the semester were a clear winner followed by day workshops during the summer and before start of semester. Weekend sessions were not popluar at all. Single day workshops were preferred.

Next year it seems that the majority of respondents were planning to use Moodle in both fall and spring semesters indicating that their experience was one which they wanted to repeat! Only 10% of responents didn't plan to use Moodle at all.

go to top of page

Conclusions

To be concluded ...

go to top of page