- Title
- Exams in computer programming: what do they examine and how complex are they?
- Creator
- Simon,; Sheard, Judy
- Relation
- 2012 Australasian Association Engineering Education Annual Conference (AAEE). Proccedings of the 2012 Australasian Association Engineering Education Annual Conference (Melbourne 3-5 December, 2012)
- Publisher
- Australasian Association for Engineering Education Conference (AAEE)
- Resource Type
- conference paper
- Date
- 2012
- Description
- Background: Computer programming is taught to students of engineering, IT, science, and other disciplines. It is taught using many different programming languages, and with no clear agreement as to what material should be covered and to what depth. We are investigating the exams in introductory programming courses to explore what they have in common and how they differ. For each question in an exam we note the topics covered, the overall difficulty of the question, and six different measures of question complexity. Purpose: Our purpose is to acquire an understanding of the variability to be found in computer programming exams, with a view to establishing the extent to which they can be said to be assessing the same thing, and the extent to which they share a similar level of complexity. Design/Method: We analysed every question in 15 introductory programming exams using a set of measures designed for the purpose in light of the literature that we were able to find on related topics. The classifying was carried out by a team of academics after an inter-rater reliability test to ensure that the members of the team were classifying with reasonable agreement. Results: We have found clear variation in the exams that we studied. Most have a good mix of easy, moderate, and hard questions, although a third of them have no questions that we considered hard, and in two exams half or more of the marks are for questions that we considered hard. All of the other complexity measures correlate with overall question difficulty, suggesting that a number of different factors can contribute to a question being considered hard. Conclusions: This study finds that, beyond the inevitable variation in exams, it can reasonably be concluded that introductory programming exams are assessing the same topics, and doing so with questions at comparable levels of complexity. The study also serves to make examiners aware of the distinction between ‘good’ complexity, such as questions involving inherently complex tasks, and ‘bad’ complexity, such as unnecessarily convoluted questions.
- Subject
- computer programming; assessment; examination; question complexity; question difficulty
- Identifier
- http://hdl.handle.net/1959.13/1319648
- Identifier
- uon:23926
- Identifier
- ISBN:9780987177230
- Language
- eng
- Reviewed
- Hits: 1131
- Visitors: 1412
- Downloads: 0
| Thumbnail | File | Description | Size | Format |
|---|