• Successful pilot trial in Spring 04 with 2 courses [Moodle evaluation : spring 04]
• Little publicity for Fall interim rollout. Plan was for a low key startup [ITPC report to the faculty]
• Instead there was massive interest from faculty
• Tapped pent-up demand for online delivery of course material
• Demand for workshops over Christmas break [Mark's IT blog archives]

• The usage made of the Moodle system was assessed by examining all the courses in the system and collecting statistics (handout).
  A summary of these data was presented at the Glitter meeting at Hope College in spring 05.
• An online survey of faculty opinions was performed in Fall 04 and again in Spring 05
• An online survey of student opinions was performed in Fall 04.
• In addition, students who were using both Moodle and Chef were surveyed about their opinions.

Despite the deliberate low key initial roll out of Moodle there was a lot of interest from faculty which prompted some questions:

1. What prompted faculty to want to use Moodle ?
2. How many faculty used Moodle in Fall & Spring semesters ?
   Some courses were created but not used — what proportion, and did this change between semesters ?
3. What were the reasons for courses not being used ?
4. Closer examination of data revealed considerable “churn” in faculty numbers between Fall & Spring semesters. What was going on here ?
5. What were the reasons behind this turnover ?
6. Finally, the number of courses taught per faculty would also give a good indication of enthusiasm.

• Most popular reason was making class materials available online
• Ease of use and peer influence were also important.
• Desire to have students engage in online work was less apparent.
• Willingness to experiment with online course management.
• Comments such as “Moodle rescued me” and “INDISPENSIBLE” indicated a high level of enthusiasm.

• Recall that only two courses had participated in the pilot trial.
• 32 courses used Moodle in the Fall semester & 47 in the Spring.
• Indicates a well-spring of interest & enthusiasm.
• But out of 100 courses created 21 were not used
• Similar proportions for Fall & Spring semesters
• Unused courses indicate rush of enthusiasm but no follow through

• Major reasons were lack of time to learn Moodle or scan resources.
• Negative reasons were not apparent.

• Moodle usage data was examined more closely to track individual faculty leaving & adding.
• 17 faculty who used Moodle in the fall chose not to use it in the following spring semester.
• Only 10 faculty who had used Moodle in the fall continued to use it in spring courses
• 23 new faculty came on board.

• Generally, positive reasons were given for not using Moodle again in the spring semester.
• Negative reasons to do with excessive time take to scan material or a dissatisfaction with Moodle in general were not given. An encouraging result.

• The majority of faculty tried Moodle with a single course
• However a considerable proportion had two courses in Moodle
• Some faculty even hosted all of their course load in Moodle.
  (The MAT program used 4 courses in the fall but none in the spring)

Research studies indicate that the usefulness of a course management system is directly related to its ease of use [comment in IT blog]. Questions:

1. Did faculty find it easy to get up & running with EC's Moodle and was it intuitive to use (ie did the system feel 'natural' rather than contrived)?
2. What were faculty's perceptions of the time they spent getting to grips with the Moodle system?
3. Did students think Moodle easy to use & what did faculty perceive their students felt about ease of use?
4. Was Moodle worth the effort? Did faculty feel that it made a difference to their course?
5. Did students feel that Moodle was an asset to the course and helped the learning process?

• Nearly all faculty agreed that it was easy to get up & running with Moodle
• 57% found Moodle intuitive to use; 47% did not. Need for intentional training.

• Most faculty spent a reasonable amount of time getting to grips with Moodle.
• Few felt that they spent too much time, and no-one felt it was inordinate.
• Conclude that Moodle is easy to pick up especially if you are already comfortable with web based applications.

Student opinions

Faculty opinions

• Most faculty asserted that their students found Moodle easy to use
• Some strongly disagreed however.
• But the vast majority of students surveyed found Moodle straightforward to use.

• Most faculty cited Moodle's usefulness in managing the course rather than effect on pedagogy.
• But significant proportion claimed that it helped the course pedagogy as well as management.
• Few suggested that it took too much effort, and no-one claimed that it was a waste of time

• Vast majority of students agreed that Moodle was an asset to the course (Q4)
• 50% said that Moodle was a useful part of the course but not essential (Q6)
• 25% asserted that it greatly enhanced their learning
• Only a very small proportion had a negative view.

How was Moodle actually used in practice and what were student's reactions?

1. The impact of Moodle on the campus was considerable with over half the students taking at least one course involving Moodle (3:1)
2. Which course outline format was the most popular (3:2), how did students like this (3:3)?
3. Which Moodle Activities were used by faculty (3:4)?
4. Online Resources comprise the course content. How much were these used (3:5), what types were employed (3:6) and how did students use resources in practice (3:7)?
5. How much was the Assignment activity used (3:8), were they online, offline or both (3:9) and did students want to submit assignments online (3:11)?
6. What were student views of getting feedback and grades for assignments online (3:11)?
7. How did students actually use Moodle in practice (3:12)?
8. What were some unorthodox uses of our Moodle system (3:13)?

• Numbers of faculty and courses represent unique instances
• However, the numbers of students in courses using Moodle are aggregated — many students were enrolled in more than one course. Therefore, number of unique students would be a lot lower.
• Nevertheless, the “reach” of Moodle across the student population on campus was considerable — we can safely say that over half the students on campus used Moodle in a course.
• Average size of class similar from fall to spring semesters

• Moodle's course Outline allows faculty to customise the way their course is presented.
• Blocks of content can be grouped as weeks (Weekly) or arbitrary topics (Topics)
• The Social outline is basically a single discussion forum.
• Weekly & Topics format equally popular.

• The outline format of a Moodle course allows faculty to scaffold activities around the syllabus contents
• This has strong pedagogic value and the students agreed that it was a rather helpful feature.

• Resources (uploaded files, web pages, web URLs) were used by nearly every course. These basically encapsulate the course content.
• News forum used to make announcements
• Assignments could either be online or offline
• Forum was the name given to discussion forums
• Journal used for online writing assignments
• Choice was used for polling students, eg to select groups to join.
• Glossary used in creative way by Maths faculty
• The Quiz activity requires work at the front end but is very flexible.

• Most courses used fewer than 15 resource items (40 courses, 50%)
• Surprise #1: number courses with 20-30 resources (20 courses, 25%)
• Surprise #2: number courses with greater than 50 resources (6)
• This is a measure of intensity of use and indicates that Moodle was being used heavily in over half the courses

• Adobe Acrobat files and Web links were the most popular of resource types
• MS Word was also popular but caused problems for students
• Powerpoint also caused problems - many converted to Acrobat format
• Text & HTML pages inside Moodle also used
• Links to personal web sites indicate “power users”
• “Directory” is a useful container for making a bunch of files available.

• 10% of students surveyed made no use of these resources (even when told to)
• 26% only used the resources when explicitly told to in class or via email by the faculty
• 19% used the resources on their own initiative to do assignments (as well as when told to)
• 42% used resources to revise for a quiz or do assignments
• A mere 3% checked resources regularly to see what had been updated.

• Assignments can be either offline — description only, student hands in paper assignment
• Or online - upload a single file
• Assignments can have online grades and feedback
• the chart shows that faculty made moderate use of assignments in their courses

• Usage of online submission of assignments was slightly less popular than offline
• A quarter of courses with assignments used both online and offline together.

Of students in courses that did not use online assignments, 50% considered that they would like to submit assignments online.

• All students agreed that getting feedback to online writing assignments (Journal entries) was helpful and over a third strongly agreed (Q10).
• All students agreed having grades presented online within Moodle was helpful and over a quarter strongly agreed (Q11)

Students did not generally use initiative in checking their course on Moodle for updates made by faculty.

• 8% only checked the course when an announcement was sent via email. Postings to the News forum are automatically emailed to the students.
• 6% of students only checked Moodle when explicitly told to do so in class.
• Half of students polled also checked their Moodle course when they knew they needed to do an assignment.
• A further 30% checked it occasionally and only 5% checked the course every day regardless.

• Senior Seminars — a group of seniors become 'teachers' for their senior seminar course on Moodle
• Students uploading resources — a group of students undertook to scan in material and upload as resources for the whole class.
• Wiki for individual & group web sites — student course web sites constructed with Wiki activity. Enables group web sites.
• Ploughshares shared course with students from Manchester and Goshen.
• Committee / consultation, eg Admissions, Fullbright Fellowship, Dialogue, Moodle Workshop.

Here we cover some written comments from faculty (4:1) and students (4:2).

Moodle 'best practices' (4:3) is an area where collaboration within MITC but also the wider Moodle community would reap benefits.

Where do faculty want to go from here (4:4)? There are some exciting developments in the Moodle world. As of June 05 version 1.5 is available with many new features. Version 1.6 should be here soon which will have a built in blogging system. With a year's experience under their belt many faculty are poised to push the envelope with their use of Moodle (4:4).

Taken from Fall and Spring surveys

• “I love being able to use it and have students using it.”
• “how to motivate to use the site is biggest challenge.”
• “I realized too late that I needed to use several copyrighted articles that I had not covered in course fees. Moodle rescued me!”
• “I used it in a F/K as a clearinghouse for the work of the whole group. INDISPENSABLE!”
• “For me, the larger the class the more use I found for Moodle”
• “I only used it mostly to post homework asssignments on the web, however this had a small but noticeable affect on how smoothly the course ran.”
• “Moodle is a great resource to have available. We badly need the flexibility it gives us.”

Taken from Survey of Student Opinions of Moodle

• “I found it very easy to use without a lot of instructor guidance, and I'm not very good with computers”
• “It was SUCH an asset to have access to all of her lectures, images, and additional resources”
• “If it weren't for powerpoints on Moodle, I would have failed numerous tests!”
• “The approachable quality of class content online was what i liked”
• Which aspect of Moodle did you like the least : “The fact that my late assingments wouldn't be graded.”
• “assignments - submitting with this system was easy and useful, grades - nice to be able to check and keep up with status”
• “My favorite part of moodle was definitely the profile option. I found it so liberating to post a photograph to represent my online persona! I also enjoyed viewing the "avatars" of others, not to mention being able to see when the last time everyone logged on was. ”
• “Best: Didn't have to buy books Worst: Wasted a lot of paper from the computer lab”

From the pattern of system usage and the surveys administered we can draw some general conclusions about best teaching practices:

• Faculty need to be intentional in telling students to check Moodle for assignments, resources, etc (slide 3:7, slide 3:12). The News forum is the most effective way of doing this but only 50% of courses used News (slide 3:4). “What we have here is a failure to communicate”. Clearly, faculty need to acquire a habit of announcing changes to their Moodle course via the News forum.
• Online feedback to assignments and online grades are very popular with students (slide 3:10). Students really appreciated knowing how well they were doing and getting written feedback to their work. Although the gradebook is rather crude in Moodle 1.4 we can expect big changes in the upcoming version 1.5.
• Generally, Moodle worked best with a class when online events, such as new resource uploaded or new assignment, were integrated into the classroom teaching rather than being merely an appendix on the course.

• Faculty were keen or ultra keen to find out more about News forum, Assignments, Forum, Journals,
• There was less interest in Blogs, choice or groups
• There was a surprising amount of interest in pedagogical tools such as workshop