Louise Dennis: Teaching Portfolio

Group Project Appraisal

The group project undertaken by my Learning Set was a survey of automated methods of assessment. There were four group members and between us we surveyed three different technologies in detail (essay marking software, program marking software and MCQs). Our conclusions were, perhaps, predictable in that we found the essay marking software to be universally unsatisfactory while we felt that automated markers for programs and MCQs could be used profitably providing they were used wisely and in appropriate contexts.

I don't think, as a group, we were particularly surprised by the outcomes and in a direct way I don't think the outcomes had any real influence on my teaching practice except that it reinforced and upheld certain assumptions.

However, that said, there were two areas in which the outcome did have an impact. Firstly, when surveying briefly the use of automated marking for mathematical problems I came across a system which asked students to generate examples of mathematical concepts (The AIM System [Sangwin, 02]). This was well received by students and appeared to encourage deep learning rather than the by rote recall of definitions or procedures. While I've not integrated such a system into the teaching of G51MC2 I have used the construction of instances increasingly in exercise and exam question setting.

Secondly I was largely unaware of the literature on MCQs, investigated by Paul Smith, and in particular in their uses for evaluating question quality. I was also impressed by the scope they offered, if used widely, for guaranteeing fairness from year to year. Currently the School of Computer Science and IT has no access to MCQ marking facilities and so I have not been able to really put this into practice. However I now believe that MCQs have a useful and important place in the range of techniques used for assessment (whereas previously I considered them largely a vehicle for surface learning) and if an easy route were presented in future would certainly attempt to incorporate them into my own teaching.

The feedback from the presentation on the project doesn't come in a form which can be trivially translated into averages for different aspects of the project. The majority of the feedback forms contain mostly "acceptable" ratings for each area with a smattering of "highs". In all only 5 of the forms rate any area "low" and three of these rank "Range and depth of data sources" in this category. Turning to the actual comments on the sheets several point out that there is little mention of the literature and this feature was ranked "low" by one of the official assessors. I feel these two issues are linked. It was not really customary in any of the disciplines involved in the project (which by the time we reached the presentation were only Computer Science and Biomedical Science) to include references in presentations. In fact in Computer Science nearly all presentations are accompanied by a full paper to which the audience may refer for references. All our references were, in fact, delegated to the website we produced as part of the project and we hadn't really appreciated that we would be assessed on the presentation alone, not on the project as a whole. Reading comments on the sheets relevant to the issue of data sources, it is clear that the major complaint was that we had not made it clear whether we had done any experimental work ourselves, which we had not. The project was intended to be a literature survey. In retrospect it is easy to see how discipline assumptions about how presentations are approached made us overlook the fact that in a presentation about a literature survey citations needed to prominent on the lecture slides and in fact that heavier signposting of the literature would have alleviated concerns about where and how any data had been obtained.

References

[Sangwin, 02] C. J. Sangwin. New opportunities for encouraging higher level mathematical learning by creative use of emerging computer aided assessment [Online] http://www.mat.bham.ac.uk/C.J.Sangwin, 2002.