Many thanks to everyone that completed the online questionnaire.
The tables presented here summarise some of the initial findings. A detailed analysis will be contained within the working group report.
Table 1 presents a summary of respondents by country of origin, gender and topic taught. 35% of respondents are female, 41% originate within the UK and the most commonly taught topic is programming (25%).
|
Topic |
Country |
Totals |
|||||||||||
|
Australia |
Finland |
UK |
USA |
Other |
|||||||||
|
M |
F |
M |
F |
M |
F |
M |
F |
M |
F |
M |
F |
Total |
|
|
Programming |
1 |
3 |
6 |
|
8 |
2 |
6 |
3 |
2 |
|
23 |
8 |
31 |
|
Mathematics |
1 |
|
|
|
1 |
|
|
|
|
|
2 |
0 |
2 |
|
Computing theory |
|
|
|
|
|
1 |
1 |
1 |
|
|
1 |
2 |
3 |
|
Data structures/algorithms |
|
1 |
3 |
|
2 |
|
1 |
2 |
1 |
|
7 |
3 |
10 |
|
Databases |
|
|
1 |
2 |
1 |
3 |
1 |
1 |
|
1 |
3 |
7 |
10 |
|
Information systems |
1 |
|
1 |
|
3 |
1 |
|
|
1 |
|
6 |
1 |
7 |
|
Systems analysis |
3 |
1 |
|
|
2 |
2 |
|
|
|
|
5 |
3 |
8 |
|
HCI |
|
2 |
1 |
1 |
1 |
3 |
|
|
|
1 |
2 |
7 |
9 |
|
Networks |
1 |
|
|
1 |
3 |
3 |
|
|
|
|
4 |
4 |
8 |
|
WWW |
|
|
2 |
|
3 |
|
|
|
|
|
5 |
0 |
5 |
|
Hardware/computer architecture |
|
|
1 |
1 |
2 |
1 |
|
1 |
|
|
3 |
3 |
6 |
|
Operating systems |
1 |
|
|
|
|
|
4 |
|
|
|
5 |
0 |
5 |
|
Real-time systems |
|
|
|
|
1 |
|
|
|
|
|
1 |
0 |
1 |
|
Distributed systems |
|
|
|
|
1 |
1 |
|
|
|
|
1 |
1 |
2 |
|
Design |
|
|
1 |
|
1 |
1 |
|
|
|
|
2 |
1 |
3 |
|
Ethics |
|
|
|
|
|
|
1 |
|
|
|
1 |
0 |
1 |
|
IT |
|
1 |
1 |
|
1 |
1 |
|
|
|
1 |
2 |
3 |
5 |
|
Compilers |
|
|
|
|
1 |
|
|
|
|
|
1 |
0 |
1 |
|
Ubiquitous computing |
|
|
1 |
|
|
|
|
|
|
|
1 |
0 |
1 |
|
Software Engineering |
2 |
|
1 |
|
|
|
|
|
|
|
3 |
0 |
3 |
|
Data Mining |
1 |
|
|
|
|
|
|
|
|
|
1 |
0 |
1 |
|
Totals |
11 |
8 |
19 |
5 |
31 |
19 |
14 |
8 |
4 |
3 |
79 |
43 |
122 |
|
19 |
24 |
50 |
22 |
7 |
|
||||||||
Table 1
Table 2 shows the percentage of academics teaching a particular topic and setting an assessment of the type stated. It is often the case that more than one assessment (and assessment of different types) is set by any particular academic during any teaching period.
|
|
Assessment type |
|||||||
|
Essay |
Other written exercise |
Practical work |
Closed book exam |
Open book exam |
In-class test |
Presentation |
Other |
|
|
Overall
|
25% |
55% |
74% |
57% |
16% |
31% |
34% |
17% |
|
Programming |
16% |
45% |
90% |
68% |
19% |
35% |
19% |
23% |
|
Mathematics |
|
50% |
50% |
100% |
|
50% |
|
|
|
Computing theory |
|
33% |
33% |
100% |
33% |
67% |
|
|
|
Data structures/algorithms |
20% |
60% |
60% |
60% |
30% |
40% |
10% |
30% |
|
Databases |
30% |
70% |
80% |
50% |
10% |
40% |
50% |
|
|
Information systems |
43% |
29% |
71% |
43% |
|
14% |
71% |
14% |
|
Systems analysis |
|
75% |
75% |
75% |
|
13% |
63% |
13% |
|
HCI |
33% |
67% |
78% |
33% |
33% |
|
33% |
22% |
|
Networks |
25% |
63% |
75% |
50% |
13% |
38% |
50% |
25% |
|
WWW |
40% |
20% |
100% |
40% |
|
|
20% |
|
|
Hardware/architecture |
17% |
50% |
67% |
67% |
|
67% |
17% |
33% |
|
Operating systems |
60% |
60% |
60% |
80% |
40% |
80% |
60% |
|
|
Real-time systems |
100% |
|
100% |
100% |
|
|
100% |
|
|
Distributed systems |
50% |
100% |
100% |
50% |
|
|
|
50% |
|
Design |
|
33% |
67% |
|
33% |
|
33% |
33% |
|
Ethics |
|
100% |
|
100% |
|
|
100% |
|
|
IT |
60% |
80% |
20% |
40% |
|
60% |
20% |
|
|
Compilers |
|
|
100% |
|
100% |
|
|
|
|
Ubiquitous comp |
|
100% |
100% |
|
|
|
100% |
|
|
Software Engineering |
|
67% |
67% |
67% |
|
|
33% |
33% |
|
Data Mining |
100% |
|
|
|
|
|
100% |
|
Table 2
Table 3 summarises the submission mechanisms and marking regimes employed by respondents to the survey. It is notable that the majority of submissions in all categories except practical work are manual and an overwhelming majority of respondents employ manual marking.
|
|
Essay |
Other written exercise |
Practical work |
Closed-book exam |
Open-book exam |
In-class test |
Presentation |
Other |
|
|
Submission
mechanism |
Manual |
21 |
46 |
46 |
63 |
10 |
28 |
28 |
6 |
|
Electronic |
13 |
32 |
54 |
10 |
8 |
11 |
22 |
12 |
|
|
Marking
techniques used |
Manual |
26 |
47 |
55 |
58 |
11 |
27 |
24 |
5 |
|
Pt
manual, pt electronic |
4 |
14 |
26 |
10 |
5 |
6 |
6 |
4 |
|
|
Electronic |
1 |
5 |
11 |
5 |
4 |
6 |
5 |
6 |
|
|
Peer
assess |
3 |
10 |
16 |
1 |
1 |
1 |
12 |
2 |
|
|
Interview |
2 |
4 |
16 |
0 |
1 |
1 |
10 |
5 |
|
Table 3
The different styles of assessment task have been split into their taxonomic components (following Bloom’s taxonomy) and the results are shown in Table 4. Whilst the majority of assessments assess understanding and application it is the case that all 5 of the levels are addressed at some stage.
|
|
Essay |
Other written exercise |
Practical work |
Closed book exam |
Open book exam |
In-class test |
Presentation |
Other |
|
Remembering |
5 |
12 |
20 |
55 |
8 |
20 |
5 |
5 |
|
Understanding |
23 |
49 |
62 |
62 |
17 |
33 |
25 |
11 |
|
Application |
17 |
39 |
81 |
39 |
12 |
19 |
23 |
10 |
|
Problem
solving |
13 |
41 |
71 |
41 |
11 |
20 |
11 |
7 |
|
Evaluation |
19 |
26 |
40 |
25 |
9 |
13 |
23 |
8 |
Table 4
Plagiarism is an ongoing and increasing problem. Table 5 shows a breakdown of academics’ perceptions of the problem within CS by assessment type.
|
|
Essay |
Other written exercise |
Practical work |
Closed book exam |
Open book exam |
In-class test |
Presentation |
Other |
|
Not
at all |
30% |
20% |
25% |
80% |
60% |
73% |
59% |
36% |
|
Minor
problem |
38% |
47% |
47% |
16% |
24% |
15% |
27% |
36% |
|
Moderate
problem |
24% |
23% |
19% |
3% |
12% |
7% |
11% |
7% |
|
Major
problem |
5% |
6% |
5% |
1% |
0% |
2% |
0% |
0% |
|
Don’t
know |
3% |
5% |
5% |
0% |
4% |
2% |
3% |
21% |
|
Total |
37 |
66 |
88 |
70 |
25 |
41 |
37 |
14 |
Table 5
Table 6 shows the proportion of respondents that have, or have colleagues who have, used or do use CAA within their teaching.
|
|
No |
Yes |
Total |
||
|
A little |
A lot |
||||
|
Do
you have any experience (past or present) of using online learning environments? |
36% |
64% |
116 |
||
|
Have
you ever used computer-aided assessment? |
38% |
41% |
21% |
117 |
|
|
Is
computer-aided assessment used within your department? |
26% |
59% |
15% |
117 |
|
Table 6
Table 7 shows a summary of the opinions held by CS academics with regard to the perceived effects of CAA on different aspects of assessment and student learning. The majority of respondents agree that CAA saves time, reduces marking time, allows greater objectivity, and improves the immediacy of feedback to students whilst allowing them to work at their own pace, but are unconvinced that CAA can help with the plagiarism problem.
|
|
Strongly disagree |
Disagree |
Neutral |
Agree |
Strongly agree |
Total |
|
CAA
has fewer security risks than manual assessment |
5% |
32% |
40% |
15% |
7% |
114 |
|
CAA
is more time-consuming than manual assessment |
10% |
35% |
28% |
21% |
5% |
113 |
|
CAA
reduces marking time |
2% |
4% |
12% |
50% |
32% |
114 |
|
It
is possible to test higher-order learning using CAA |
4% |
27% |
33% |
33% |
3% |
114 |
|
CAA
offers greater objectivity in marking |
3% |
19% |
25% |
45% |
9% |
113 |
|
CAA
allows students to work at their own pace and more flexibly |
1% |
10% |
26% |
45% |
18% |
114 |
|
The
use of CAA makes students more anxious |
2% |
25% |
54% |
18% |
2% |
112 |
|
CAA
improves the immediacy of feedback to students |
0% |
4% |
5% |
64% |
26% |
114 |
|
CAA
improves the quality of feedback to students |
5% |
29% |
42% |
19% |
5% |
111 |
|
CAA
disadvantages special-needs students |
4% |
28% |
52% |
13% |
2% |
113 |
Table 7
Back to working group main page.