CHEF : Management courses Spring 2004

Students from courses in the Management program taught by Monteze Snyder (210 (Human Relations and Organizational Decision Making - 37 students), 344 (Strategic Planning And Marketing - 19 students), 488 (Senior Seminar - 3 students) ) were asked to complete an online assessment hosted at the Free Assessment Summary Tool site (password 'chef').

Most of the questions listed below were formulated as a statement to which students used the Likert scale of 5 points of agreement to disagreement to respond. The 'Not Applicable' option was taken to indicate a neutral position between Somewhat Agree and Somewhat Disagree. Question 8 was multiple choice.

A total of 45 individual students (many were in two or more of these courses) were asked to complete the survey; 22 responded. While this is not a bad return rate it's quite likely that those students who felt strongly one way or the other about CHEF would register their views while those who were tepid or neutral would be less inclined to do so (the survey was voluntary). The consequences of this will be elaborated upon in the Discussion section.

The table below presents the questions asked (together with the options for the multiple choice question) and the totals responding for each Likert category.

Question 12 called for a free text answer:
Can you think of other ways that CHEF or other CMS might enhance learning, eg, online quizzes.

The results presented were charted in MS Excel as pie charts to show the percentage of the total that each response took.

1. Going in to the class, most students were comfortable with using computers. There seemed to be none who were completely at sea.

2. 23% (5 students) had used a CMS before - probably Course Compass in a Chemistry class.

3. Half of the students who had used a CMS before considered that CHEF was better than the other CMS, and half thought that the other CMS was superior. Given that the other CMS was probably Course Compass (using Blackboard) this suggests that CHEF holds up well compared to the more polished commercial offerings.

4. Four tools were consistently used in these courses; Resources, Discussion, Assignment and Chat.

   Students found the that resources tool was the easiest to use with 86% agreeing with the proposition and only 9% in disagreement.

5. The discussion tool ranked lowest in student's estimation of ease of use with 64% in agreement that it was easy and straightforward to use, 18% lukewarm, and 18% disagreeing.

6. Opinions about the assignment tool were most polarised with 23% of students disagreeing somewhat that this was easy to use but 9% strongly agreeing.

7. The chat tool was universally liked, with only 9% of students somewhat disagreeing that it was easy to use.

   Clearly the resources and chat tools were deemed the easiest of the CHEF tools to used which is perhaps unsurprising since these were also the simplest to use. Where the nature of the tool demanded more student interaction, as with discussion and assignment tools, there was a greater spread of opinion. The discussion tool seemed to leave more students unimpressed (18% were neutral about its being easy to use, and there were no students in strong agreement) and the assignment tool had the greatest proportion of students who thought it was not very easy to use (23%). However, what is remarkable is that there were no students at all who strongly disagreed with any of the tools being easy to use. This does indicate that on the whole CHEF was reckoned to be reasonably usable by the students.

8. The responses to the question were among the most interesting of the survey. We expected more of a positive response to the question of whether CHEF helped or hindered learning. Instead we found a polarized response with 23% of students (5 individuals) claiming that they were learning in spite of using CHEF and 9% (2 students) claiming that CHEF made the class more difficult. Only 54% of respondents considered that the use CHEF helped their learning. As noted in the introduction the overall return rate to the survey was rather low (22 students from a total of 45) and consequently could magnify the extremes of opinion. Even so, the low proportion of students who seemed to be enthusiastic about CHEF assisting their learning was rather disappointing.

9. Responses to this question throw some light on results from question 8. There was even greater polarization of opinion here, with a quarter of the class either strongly or somewhat agreeing that CHEF was being used effectively in class, and just over a third disagreeing. It's likely that the same students who considered that CHEF was being used effectively were also of the opinion that it enhanced their learning, and vice versa. Moreover, it's also likely that those students who were doing less well in class would be more inclined to give a negative response to this question.

   On further consideration, the responses to questions 8 & 9 which have to do with class experience and CHEF are really showing that some students are very likely not making objective responses and that what we are seeing is a minority of students having a hard time with the class as a whole.

10. Here again, the responses were somewhat surprising. Over a third of the students disagreed (18% strongly, 18% somewhat) with the proposition that getting feedback to assignments was helpful, a third were neutral, and a third somewhat agreed. There was no student who strongly agreed. What can one say apart from wondering whether this had anything to do with the way CHEF worked or rather whether it was a result of the content of the feedback?

11. Similarly, one would expect students to be enthusiastic about having grades available, but the 40% of students who didn't seem to care is baffling, likewise the 9% claiming that having their grades in CHEF was unhelpful.