2007 Reports
e-Learning Enhancement
Mrs Rosemary Kardos (Oral Sciences)
ePortfolio Project
The requirement for health practitioners to collect evidence for certification under the Health Practitioners Competence Assurance Act 2003 has created an opportunity to use industry requirements to inform practices for emerging professionals. The concept of an ePortfolio is seen as a logical approach to meeting this requirement. Individuals have access to and control over the capture, storage, sharing and publishing of evidence of their professional abilities and experience. The first phase of this project has been to design and construct a comprehensive Professional Practice Portfolio capable of meeting the needs of students, staff and associated professional bodies. A web-based portfolio system has been developed and is currently being trialled by staff and students involved in the Bachelor of Oral Health degree. It is hoped, in the second part of this project, that we are able to illustrate working processes that allow users to capture, organise, collaborate, share and publish details of their professional skills, attributes and experience effortlessly. While our primary interest has been to work with students, we believe further development of the application could also provide other health professionals with a superior solution to generating professional portfolios that meet the requirements of their own Professional Councils and Boards.
Professor Kwok-Wing Lai (UO College of Education)
Learning resources to support distance online students
Increasing demand for online distance education together with international research suggesting disturbingly low completion and satisfaction rates have highlighted a need for the increased support of distance students. These support needs have been found to include: increased social presence of students and staff, and students’ time management and online study skills.
This project developed and gathered resources of this nature (including those previously developed as part of the OIL project available at http://oil.otago.ac.nz/oil/ and course specific videos of lecturers, course coordinators, remote librarians and ITS staff) for use by students enrolled in distance papers with the College of Education. These resources were trialled with one 300 and one 400 level class. Student responses suggested that when accessed, the resources were seen as being very useful, however students needed more encouragement to make use of these resources. Additional resources have continued to be developed and added to the course website to provide support to students.
Associate Professor Jim Reid (General Practice, Dunedin)
An investigation of pedagogically and technically appropriate virtual learning and collaboration environments with a view to recommending enhancements to existing ICT tools for learning and teaching within the University of Otago
There are many terms that describe the employment of assorted technologies via the internet (or local intranet) in order to assist and better learning, both in terms of performance and knowledge. This integration of ICT with learning theory has been popularly coined 'e-learning or 'web based learning', (WBL).
The effective use of these novel technologies should optimise learning and should also justify the considerable investments in time and money that are required to properly establish them. To achieve this it follows that our use of educational technologies must be guided by established educational theory.
E-learning has been heralded as a unique tool that can assist in the establishment of modern pedagogical models in particular by enabling individualised learning while promoting an interactive role for the educator. In a web based learning environment the educator can act as a facilitator of learning, as opposed to a mere disseminator of content that is the role sometimes ascribed to instructors in more traditional models.
While there is much discussion regarding how best to implement the various new tools that e-learning offers, the increasing number of publications relating to educational technologies highlights the need for evidence relating to when, how and which option to employ.
Dr Terry Scott (Physics)
An Online Tutoring System for Problem Solving in Physics
The purpose of this project was the design and implementation of an integrated learning and assessment tool based on the PAL (Personal Assistants for Learning) system developed by the Centre for Innovation in Learning at Carnegie Mellon University which has been shown to be almost as effective as one-on-one tutorials with an experienced tutor. The OUPAL tool consists of an Adobe Flash based user interface, an online library of reference material, a library of problems, and a database. It has been designed to be extensible by physics staff both in terms of content and interaction with the system.
The reciprocal tutoring aspect of the project, a key part in promoting specific problem solving strategies, is provided by a set of general problem ‘types’, which are available to OUPALs authors. These correspond to the Coaching, Implementation, and Performance problem styles of the detailed in the grant proposals which provide, and model, problem solving strategies. The close association of problems with reference material, up to and including the inclusion of searches through reference material included in a problem, encourages students to utilise the course material in a problem solving context.
The OUPALs system collects data on how a student responds to any problems they may attempt, allowing this data to be used to track an individual student’s development, and assess the success of the course being taken as a whole. This system allows the training of students in effective problem solving by the provision of specific problem solving strategies and reinforcing the use of these strategies through guided practice using the coaching/implementation problem types. In addition the system allows for the collection of detailed data concerning the development of student problem solving skill.
ePortfolio Project
The requirement for health practitioners to collect evidence for certification under the Health Practitioners Competence Assurance Act 2003 has created an opportunity to use industry requirements to inform practices for emerging professionals. The concept of an ePortfolio is seen as a logical approach to meeting this requirement. Individuals have access to and control over the capture, storage, sharing and publishing of evidence of their professional abilities and experience. The first phase of this project has been to design and construct a comprehensive Professional Practice Portfolio capable of meeting the needs of students, staff and associated professional bodies. A web-based portfolio system has been developed and is currently being trialled by staff and students involved in the Bachelor of Oral Health degree. It is hoped, in the second part of this project, that we are able to illustrate working processes that allow users to capture, organise, collaborate, share and publish details of their professional skills, attributes and experience effortlessly. While our primary interest has been to work with students, we believe further development of the application could also provide other health professionals with a superior solution to generating professional portfolios that meet the requirements of their own Professional Councils and Boards.
Professor Kwok-Wing Lai (UO College of Education)
Learning resources to support distance online students
Increasing demand for online distance education together with international research suggesting disturbingly low completion and satisfaction rates have highlighted a need for the increased support of distance students. These support needs have been found to include: increased social presence of students and staff, and students’ time management and online study skills.
This project developed and gathered resources of this nature (including those previously developed as part of the OIL project available at http://oil.otago.ac.nz/oil/ and course specific videos of lecturers, course coordinators, remote librarians and ITS staff) for use by students enrolled in distance papers with the College of Education. These resources were trialled with one 300 and one 400 level class. Student responses suggested that when accessed, the resources were seen as being very useful, however students needed more encouragement to make use of these resources. Additional resources have continued to be developed and added to the course website to provide support to students.
Associate Professor Jim Reid (General Practice, Dunedin)
An investigation of pedagogically and technically appropriate virtual learning and collaboration environments with a view to recommending enhancements to existing ICT tools for learning and teaching within the University of Otago
There are many terms that describe the employment of assorted technologies via the internet (or local intranet) in order to assist and better learning, both in terms of performance and knowledge. This integration of ICT with learning theory has been popularly coined 'e-learning or 'web based learning', (WBL).
The effective use of these novel technologies should optimise learning and should also justify the considerable investments in time and money that are required to properly establish them. To achieve this it follows that our use of educational technologies must be guided by established educational theory.
E-learning has been heralded as a unique tool that can assist in the establishment of modern pedagogical models in particular by enabling individualised learning while promoting an interactive role for the educator. In a web based learning environment the educator can act as a facilitator of learning, as opposed to a mere disseminator of content that is the role sometimes ascribed to instructors in more traditional models.
While there is much discussion regarding how best to implement the various new tools that e-learning offers, the increasing number of publications relating to educational technologies highlights the need for evidence relating to when, how and which option to employ.
Dr Terry Scott (Physics)
An Online Tutoring System for Problem Solving in Physics
The purpose of this project was the design and implementation of an integrated learning and assessment tool based on the PAL (Personal Assistants for Learning) system developed by the Centre for Innovation in Learning at Carnegie Mellon University which has been shown to be almost as effective as one-on-one tutorials with an experienced tutor. The OUPAL tool consists of an Adobe Flash based user interface, an online library of reference material, a library of problems, and a database. It has been designed to be extensible by physics staff both in terms of content and interaction with the system.
The reciprocal tutoring aspect of the project, a key part in promoting specific problem solving strategies, is provided by a set of general problem ‘types’, which are available to OUPALs authors. These correspond to the Coaching, Implementation, and Performance problem styles of the detailed in the grant proposals which provide, and model, problem solving strategies. The close association of problems with reference material, up to and including the inclusion of searches through reference material included in a problem, encourages students to utilise the course material in a problem solving context.
The OUPALs system collects data on how a student responds to any problems they may attempt, allowing this data to be used to track an individual student’s development, and assess the success of the course being taken as a whole. This system allows the training of students in effective problem solving by the provision of specific problem solving strategies and reinforcing the use of these strategies through guided practice using the coaching/implementation problem types. In addition the system allows for the collection of detailed data concerning the development of student problem solving skill.
Innovation in Teaching
Associate Professor Catriona Hurd (Botany)
A Life Aquatic: Bringing the underwater environment into the laboratory
Subtidal habitats are inaccessible to first-hand investigation by undergraduate students due to the challenges presented when conducting research underwater. This programme introduces students to underwater habitats and seaweed species using digital video collected by divers, in combination with freshly collected seaweed specimens. Students identify key seaweed species and, using the underwater video movies, record the habitats in which they grow thereby gaining an understanding of each species’ role in a range of habitats. Students produce their own hypotheses to explain patterns of seaweed abundance across physical gradients (e.g. light, water motion) observed in videos. To test hypotheses, comparisons of percentage cover of seaweeds and other organisms were made using digital photo-quadrats taken in each habitat. The laboratory work allows students to conduct a study from initial observation and hypothesis development through to data collection, analysis and presentation. As more video and digital photographs are collected in the future, students will gain more freedom to observe and compare a wider range of habitats. This programme was successfully used in a series of laboratories and lectures for 55 students in Marine Botany (BTNY 225) in 2007 and will be used in this course in 2008 onwards.
Dr John Reynolds (Anatomy & Structural Biology)
Engaging Students in Learning in First Year Health Science HUBS Papers
The overall aim was to measure the impact of a curriculum intervention on approaches to learning in two new large courses (HUBS191 and HUBS192) which form an integral part of the first year health science programme. A further aim was to support teachers in the new curriculum and to use the data to inform ongoing teaching practice. The new courses were designed to encourage students to take a deep approach to learning. Data regarding students’ learning approaches were obtained through focus groups, and by student self-report using the Approaches and Study Skills Inventory for Students (ASSIST) inventory. The latter was administered to the BIOL115 class preceding the new courses (as baseline data), and to HUBS students at the beginning and end of the academic year. Results show that the predominant learning approach was surface, followed by strategic and then a deep approach. However, students took a deep and strategic approach to their studies to a greater extent at the end of the new curriculum, directly attributable to the curriculum intervention. The impact of a shift in learning approach is likely to become apparent as students advance in their degrees and undertake assessments aimed at examining conceptual understanding.
A Life Aquatic: Bringing the underwater environment into the laboratory
Subtidal habitats are inaccessible to first-hand investigation by undergraduate students due to the challenges presented when conducting research underwater. This programme introduces students to underwater habitats and seaweed species using digital video collected by divers, in combination with freshly collected seaweed specimens. Students identify key seaweed species and, using the underwater video movies, record the habitats in which they grow thereby gaining an understanding of each species’ role in a range of habitats. Students produce their own hypotheses to explain patterns of seaweed abundance across physical gradients (e.g. light, water motion) observed in videos. To test hypotheses, comparisons of percentage cover of seaweeds and other organisms were made using digital photo-quadrats taken in each habitat. The laboratory work allows students to conduct a study from initial observation and hypothesis development through to data collection, analysis and presentation. As more video and digital photographs are collected in the future, students will gain more freedom to observe and compare a wider range of habitats. This programme was successfully used in a series of laboratories and lectures for 55 students in Marine Botany (BTNY 225) in 2007 and will be used in this course in 2008 onwards.
Dr John Reynolds (Anatomy & Structural Biology)
Engaging Students in Learning in First Year Health Science HUBS Papers
The overall aim was to measure the impact of a curriculum intervention on approaches to learning in two new large courses (HUBS191 and HUBS192) which form an integral part of the first year health science programme. A further aim was to support teachers in the new curriculum and to use the data to inform ongoing teaching practice. The new courses were designed to encourage students to take a deep approach to learning. Data regarding students’ learning approaches were obtained through focus groups, and by student self-report using the Approaches and Study Skills Inventory for Students (ASSIST) inventory. The latter was administered to the BIOL115 class preceding the new courses (as baseline data), and to HUBS students at the beginning and end of the academic year. Results show that the predominant learning approach was surface, followed by strategic and then a deep approach. However, students took a deep and strategic approach to their studies to a greater extent at the end of the new curriculum, directly attributable to the curriculum intervention. The impact of a shift in learning approach is likely to become apparent as students advance in their degrees and undertake assessments aimed at examining conceptual understanding.
Research into University Teaching
Dr Rob Griffiths (Medicine, Wellington)
Use of Concept Mapping in Student Evaluation
The Concept Mapping project has now been concluded. The literature review into the application of concept maps for the evaluation of postgraduate medical programmes and for student assessment identified the strengths and weaknesses of the tool. Instead of seeking to use concept maps for computerised assessment, we will now integrate concept maps into our regular reviews of educational materials and as a way of building conceptual frameworks with students and conducting formative assessments of their understanding.
Dr Keryn Pratt (UO College of Education)
Effective ways of teaching large classes in an online distance environment
This research aims to determine whether using small group work in a large online class is an effective way of reducing teacher input while ensuring that interaction and learning effectiveness remain high. This was done by having students participate in both large group and small group discussions, rating the interaction and learning effectiveness of each. In 2007 students participated in discussions involving either 5-6 or 15-16 students. While insufficient numbers of students agreed to complete the questionnaires to make any formal evaluation valid, my informal observations were that in this case, 5-6 students were insufficient for any depth of discussion. Due to the lack of participation I repeated the research in 2008, this time having students participate in either whole class (approximately 30 students) discussions, or half class (15-16 students) discussions. Again, insufficient numbers of students agreed to participate in the research, however, an informal poll run using the Learning Management System Moodle found that seven students preferred whole class discussions compared with four who preferred half class discussions. I am unsure why so few students agreed to participate in the research, as this has meant I am unable to reach any final or formal conclusion.
Dr Holger Regenbrecht (Computer Science)
Non-verbal Communication Cues in Remote Teacher-Student Situations (Distant Learning)
awareness and eye-to-eye contact on the communication quality in distant teacher-student consultations using videoconferencing and virtual reality technology. Many open questions have to be answered and many technological solutions must be developed before a scenario can be implemented, which for instance allows for a remote course advising session with one student (abroad) and two advisers (at two different locations). First, we developed a solution which allows for gaze awareness with three communicating parties. Second, we empirically evaluated the influence of the visibility of the torso/hands in comparison to just the face on quality and necessary trust. Thirdly, we implemented techniques, which allow to track the hands in the desktop space to be used for remote gesturing. Currently we are investigating the influence of direct eye-to-eye contact on the communication quality.
Web page: http://www.hci.otago.ac.nz/research_CALT.html
Associate Professor Pat Shannon (Social Work & Community Development)
Development of Professional Cross Cultural Competence in Social Work
The idea of ‘cultural competence’ is slowly changing. Older approaches tended to define cultural competence as learning about the particular norms, beliefs, preferences and values of clearly and unproblematically delimited minority ‘groups’. However, the impacts of postmodern and social constructionist theorising have led to changes in this conceptualisation. This now includes awareness of political and economic forces in the production of knowledge ‘about’ cultural groups, the rapidly changing demographic landscape, and an understanding of personal agency in the multiple identities people hold. A more critical approach includes: an analysis of power; how the production of such groups can obscure important internal differences; the role of dominant groups in producing accounts of minority groups as free from wider political and economic imperatives; and how older conceptions of culture tended to impose fixed, static and one dimensional identities on already marginalised people.
The challenge in social work education is to account for this changing landscape and educate social workers accordingly. Given the fluidity of the dimensions of diversity, this social work course has been developed as a problem-based programme with diversity integrated throughout the program, rather than taught as a series of discrete topics. In the four year programme, the goal at year 1 is to make students aware of their own value and belief system. At year 2 the objectives are to impart some basic knowledge about the institutionalised forms of diversity, leading in the professional programme to skills at year 3 and the ability to incorporate them into the conceptualisation of practice in year 4.
This project attempted to find out if the education we offer is achieving these objectives, utilising a mixed methods approach to test students’ developing capabilities at each level of the social work degree program. At year 1 pre and post tests using the Quick Discrimination Index examined attitudes and found that the student scores indicated a slight increase in discriminatory attitudes. However, the qualitative comments indicated that this was because of increased awareness of student’s own positions. Above that level we tested awareness, skills and knowledge, and found small improvements in scores on the MAKKS scale in skills and knowledge in years 2, 3 and 4, but a small reduction in reported awareness. Our qualitative results showed an improved ability to conceptualise cases, and an acceptance of a ‘not knowing’ position that may account for the reduced awareness on self-report. From this perspective, student perceptions of a lack of ‘knowledge’ are in fact a positive outcome for the programme as long as they are accompanied by sensitivity and openness to the relevance of difference.
Dr Margot Skinner (Physiotherapy)
A comparison of the ability of two student cohorts to read and interpret a scholarly paper
The aim of the study was to compare the ability of a cohort of BPhty student volunteers who took the HSFY paper ENG124 to read and interpret scholarly papers with the ability of a cohort who did not take the paper. Mean results for the pooled data for the 2006 HSFY cohort and for individual groups (A-D) showed no significant difference in the critical review scores before and after the tutorial support. Mean results for the pooled data for the 2007 HSFY cohort showed no significant difference in critical review scores before and after the tutorial support. When mean pooled data for individual groups were compared there were no significant differences in the critical review scores for group D (no tutorial support) or group A (one lecture) however scores for group C (two lectures) and group B (two lectures and tutorial) were significantly improved compared to the first review (p<0 001).
Overall results suggest that students who did not take ENG 124 in HSFY benefited from the additional learning experiences in that their overall ability to read and interpret a scholarly paper significantly improved. It is thus suggested that some learning experiences involving skills for critical review should be included in Year 2 of the BPhty programme.
Dr Sarah Stein (HEDC)
Identifying knowledge and skills of experienced higher education teachers
Status: Project not completed
Use of Concept Mapping in Student Evaluation
The Concept Mapping project has now been concluded. The literature review into the application of concept maps for the evaluation of postgraduate medical programmes and for student assessment identified the strengths and weaknesses of the tool. Instead of seeking to use concept maps for computerised assessment, we will now integrate concept maps into our regular reviews of educational materials and as a way of building conceptual frameworks with students and conducting formative assessments of their understanding.
Dr Keryn Pratt (UO College of Education)
Effective ways of teaching large classes in an online distance environment
This research aims to determine whether using small group work in a large online class is an effective way of reducing teacher input while ensuring that interaction and learning effectiveness remain high. This was done by having students participate in both large group and small group discussions, rating the interaction and learning effectiveness of each. In 2007 students participated in discussions involving either 5-6 or 15-16 students. While insufficient numbers of students agreed to complete the questionnaires to make any formal evaluation valid, my informal observations were that in this case, 5-6 students were insufficient for any depth of discussion. Due to the lack of participation I repeated the research in 2008, this time having students participate in either whole class (approximately 30 students) discussions, or half class (15-16 students) discussions. Again, insufficient numbers of students agreed to participate in the research, however, an informal poll run using the Learning Management System Moodle found that seven students preferred whole class discussions compared with four who preferred half class discussions. I am unsure why so few students agreed to participate in the research, as this has meant I am unable to reach any final or formal conclusion.
Dr Holger Regenbrecht (Computer Science)
Non-verbal Communication Cues in Remote Teacher-Student Situations (Distant Learning)
awareness and eye-to-eye contact on the communication quality in distant teacher-student consultations using videoconferencing and virtual reality technology. Many open questions have to be answered and many technological solutions must be developed before a scenario can be implemented, which for instance allows for a remote course advising session with one student (abroad) and two advisers (at two different locations). First, we developed a solution which allows for gaze awareness with three communicating parties. Second, we empirically evaluated the influence of the visibility of the torso/hands in comparison to just the face on quality and necessary trust. Thirdly, we implemented techniques, which allow to track the hands in the desktop space to be used for remote gesturing. Currently we are investigating the influence of direct eye-to-eye contact on the communication quality.
Web page: http://www.hci.otago.ac.nz/research_CALT.html
Associate Professor Pat Shannon (Social Work & Community Development)
Development of Professional Cross Cultural Competence in Social Work
The idea of ‘cultural competence’ is slowly changing. Older approaches tended to define cultural competence as learning about the particular norms, beliefs, preferences and values of clearly and unproblematically delimited minority ‘groups’. However, the impacts of postmodern and social constructionist theorising have led to changes in this conceptualisation. This now includes awareness of political and economic forces in the production of knowledge ‘about’ cultural groups, the rapidly changing demographic landscape, and an understanding of personal agency in the multiple identities people hold. A more critical approach includes: an analysis of power; how the production of such groups can obscure important internal differences; the role of dominant groups in producing accounts of minority groups as free from wider political and economic imperatives; and how older conceptions of culture tended to impose fixed, static and one dimensional identities on already marginalised people.
The challenge in social work education is to account for this changing landscape and educate social workers accordingly. Given the fluidity of the dimensions of diversity, this social work course has been developed as a problem-based programme with diversity integrated throughout the program, rather than taught as a series of discrete topics. In the four year programme, the goal at year 1 is to make students aware of their own value and belief system. At year 2 the objectives are to impart some basic knowledge about the institutionalised forms of diversity, leading in the professional programme to skills at year 3 and the ability to incorporate them into the conceptualisation of practice in year 4.
This project attempted to find out if the education we offer is achieving these objectives, utilising a mixed methods approach to test students’ developing capabilities at each level of the social work degree program. At year 1 pre and post tests using the Quick Discrimination Index examined attitudes and found that the student scores indicated a slight increase in discriminatory attitudes. However, the qualitative comments indicated that this was because of increased awareness of student’s own positions. Above that level we tested awareness, skills and knowledge, and found small improvements in scores on the MAKKS scale in skills and knowledge in years 2, 3 and 4, but a small reduction in reported awareness. Our qualitative results showed an improved ability to conceptualise cases, and an acceptance of a ‘not knowing’ position that may account for the reduced awareness on self-report. From this perspective, student perceptions of a lack of ‘knowledge’ are in fact a positive outcome for the programme as long as they are accompanied by sensitivity and openness to the relevance of difference.
Dr Margot Skinner (Physiotherapy)
A comparison of the ability of two student cohorts to read and interpret a scholarly paper
The aim of the study was to compare the ability of a cohort of BPhty student volunteers who took the HSFY paper ENG124 to read and interpret scholarly papers with the ability of a cohort who did not take the paper. Mean results for the pooled data for the 2006 HSFY cohort and for individual groups (A-D) showed no significant difference in the critical review scores before and after the tutorial support. Mean results for the pooled data for the 2007 HSFY cohort showed no significant difference in critical review scores before and after the tutorial support. When mean pooled data for individual groups were compared there were no significant differences in the critical review scores for group D (no tutorial support) or group A (one lecture) however scores for group C (two lectures) and group B (two lectures and tutorial) were significantly improved compared to the first review (p<0 001).
Overall results suggest that students who did not take ENG 124 in HSFY benefited from the additional learning experiences in that their overall ability to read and interpret a scholarly paper significantly improved. It is thus suggested that some learning experiences involving skills for critical review should be included in Year 2 of the BPhty programme.
Dr Sarah Stein (HEDC)
Identifying knowledge and skills of experienced higher education teachers
Status: Project not completed