Section B: Monitoring and Review of Programmes and Modules
- B.1 Visiting Examiners
- B.2 Student Evaluation of Modules
- B.3 National Student Survey
- B.4 Annual Programme Review
- B.5 Comprehensive Departmental Review
- B.6 Portfolio Review
The appointment of Visiting Examiners, and the collection of their reports, is under the remit of the Registry. The Quality Assurance team can advise on any questions regarding these systems.
NB – the procedures below apply to Visiting Examiner participation in the examination process. These procedures apply in respect of all taught courses (including distance and online learning), as well as the Language Centre, CISD, IFCELS and the Certificate and Diploma in Asian Arts.
Each board considering candidates for degrees, diplomas or certificates awarded by the University of London or by SOAS must have at least one Visiting Examiner, being an examiner from outside the University of London. Boards may also have an Intercollegiate Visiting Examiner, being an examiner from a University of London institution other than SOAS.
The primary duty of a Visiting Examiner is to ensure that the standard of the awards of the University of London in the field of study concerned is consistent with that of the national university system. Where appointed, this is also the duty of an Intercollegiate Visiting Examiner, who has the additional duty of ensuring consistency in the standard of awards of the University of London.
Visiting Examiners oversee the setting of examination papers, have the right to inspect any examination script or other assessed examination material within the purview of the relevant board of examiners and may be asked to adjudicate on borderline cases. They attend meetings of the board at which significant decisions are taken. The views of Visiting Examiners are particularly influential in the case of disagreement on the final determination of an award.
Visiting Examiners must be persons of seniority and experience who are able to command authority. The appointment process is conducted annually. Visiting Examiners may not normally serve for more than four consecutive years, with a minimum gap of five years between each period of office (exceptions can be made by the Curriculum, Quality Assurance Committee (CQAC) where it is particularly difficult to appoint examiners, for instance in the case of languages not taught elsewhere in the UK). CQAC approves the appointment of Visiting Examiners to the School’s examination boards.
The appointment process for examiners for undergraduate and taught masters programmes is described in detail in the following documents:
- General instructions for the appointment of examiners and the conduct of examinations for first degrees of the University of London taught and examined by SOAS
- General instructions for the appointment of examiners and the conduct of examinations for postgraduate taught degrees of the University of London taught and examined by SOAS
These and other relevant documents, including an electronic version of the Visiting Examiners’ report form, can be found on the registry web pages.
Visiting Examiners are required to make a written report each year, on a form provided for that purpose by the Registry. Visiting Examiners may, if they wish, send a separate confidential report to the Director & Principal. The procedure for handling reports is as follows:
- Each report is received electronically and acknowledged by the Quality Assurance team [QA] in the Registry (or designated member of staff on behalf of QA)
- QA logs receipt of the report and becomes responsible for safe-keeping of the original and maintenance of the electronic archive.
- QA uploads the reports digitally to the BLE.
- QA alerts the Department Managers that the reports have been uploaded onto the BLE, to be discussed and scrutinised by the Department at the beginning of the next Academic session.
- QA emails the Chairs of the Sub-Boards with individual VE reports, making clear that they are responsible for addressing Sub-Board related issues raised by the VE and that programme related matters are to be addressed by programme convenors, the Head of Department and Associate Director for Teaching Quality as part of Annual Programme Review (see below).
- The Chair of Sub-Board scrutinises the report for comments (liaising with academics, Department administration and QA) and actions as required, with a report on all actions taken to be made at the next Sub-Board (updates on action taken being a standing item on each Sub-Board agenda).
- The Chair of Sub-Board provides feedback to the VE (copying in the programme convenor(s), Head of Department and Associate Director for Teaching Quality and QA), addressing issues raised and action taken.
- Programme convenors and Heads of Department address VE comments as part of Annual Programme Review (APR) (see below). As part of APR, the Associate Director for Teaching Quality makes a cumulative report to TeLSOC on points raised by VEs and action taken.
- Chair of TeLSOC to respond if necessary to VEs, liaising with Chair(s) of Sub-Board(s) on any School-wide issues identified at TeLSOC.
- Summary of major issues and action taken in response to VE reports in annual report from TeLSOC to CQAC and the Pro-Director (Learning and Teaching).
- VE reports and responses by Sub-Board Chairs to be made available (via Moodle) to students.
The Student Evaluation of Modules scheme is co-ordinated by the Quality Assurance Team.
Student feedback is a crucial part of the monitoring of modules and teaching in the School. Over the years it has led to significant improvements in module content and teaching methods.
The School regards the purpose of module evaluation as being threefold:
- to provide information to assist tutors to reflect on their module design and teaching;
- to encourage students to reflect on their own learning;
- to inform the annual monitoring of programmes.
It is therefore the position of the School that:
- Every member of teaching staff is required to monitor and, where necessary and possible, to seek to improve her or his contribution to student learning. One important means of doing so is through the systematic evaluation of modules by the students who undertake them.
- The views of students, including suggestions for possible improvements, should be obtained on all modules. At a minimum, formal student evaluations should be obtained on each module once a year by the use of Student Evaluation of Modules questionnaires. Departments may obtain evaluations more frequently and by other means (eg departmental meetings, focus groups etc) if they wish. The outcomes of this process and the effectiveness of actions taken should be fed into the annual programme review (APRs)
- Students should be informed of actions taken via both the first departmental meeting of the new session and the relevant School-wide committees.
It should be made clear to students why they are asked to evaluate modules, what happens after they have provided evaluations and how they will get feedback on the action taken as a result of their feedback. The method for obtaining module evaluation by students should be published to students. For example, procedures should be clearly stated in appropriate module documentation and/or online. Staff are also encouraged to make reference to the evaluation methods at induction events and/or at the beginning of each module.
Timescales for the completion of the process are given below and these confirm that this evaluation takes place at the end of the running of the module. This inevitably means that it will be too late to change anything for the benefit of current students, and it is therefore good practice to conduct some less formal evaluations mid-course so that action might be taken, or responses offered, in a timely manner if needed.
The Student Evaluation of Modules surveys are completed online. The surveys are then analysed, and the QA team circulated an automatically created breakdown of the quantitative responses with qualitative responses verbatim, to Heads of Department/programme convenors. Colleagues in Planning provide further detailed analysis of the data at Department and School wide levels which is shared and discussed both with Departments and at Committees.
Head of Department/programme convenor
- discusses issues raised with individual staff members where necessary
- ensures that issues raised in student feedback and action taken or planned as a result are reported to the first Departmental meeting of the academic year at which student representatives are present
- reports on issues raised in student feedback and action taken or planned as a result in Annual Programme Review
- ensures that Department/programme is supported in acting on issues raised in student feedback
- considers student feedback across the Department and reports to TELSOC, highlighting good practice identified and comments with implications for more than one programme/Department.
Quality Assurance Team
- provides data to Planning for expansive analysis, which is then shared with Department colleagues and TELSOC and other relevant groups for further discussion and action.
Student Evaluation of Modules surveys are emailed to and completed by students at the end of term 1 for term 1 modules and modules that have different lecturers in terms 1 and 2. For term 2 and full year modules, the surveys are emailed at the end of term 2. Reports for individual modules with statistical analysis and comments are emailed to convenors and Heads of Department after the surveys have ended. Summary departmental, faculty and School reports are produced by the QA Team during the summer. Indepth analysis and reports are prepared over the summer by Planning and are sent to the relevant Heads of Department and presented to the relevant committees.
A review of the methods of obtaining student evaluations and of responsiveness to student evaluations forms part of the Annual Programme Review and Comprehensive Departmental review of programmes. TELSOC will monitor the effectiveness of the policy for student evaluation and revise the procedures as appropriate.
In addition to the internal Student Evaluation of Modules, and other student surveys undertaken periodically, SOAS students participate in the National Student Survey. Introduced by HEFCE in 2005, the NSS undertakes to survey all final year undergraduates in England, Wales and Northern Ireland, publishing results online as part of the data available to prospective applicants and other interested parties. SOAS works with students to maximise their participation in the annual survey, so that representative feedback can be collected and published. The results of the NSS are reported to TELSOC and other committees annually and departments use them in planning enhancements to programmes. Programme convenors are encouraged to include consideration of this feedback along with other sources when completing their Annual Programme Reports.
Annual Programme Review [APR] of taught programmes is co-ordinated by the QA team and Department Offices, both of whom can advise on the details of the procedure, including how supporting data, student feedback etc will be made available to programme convenors. Those completing review reports should read the guidance notes before completing the review report templates. For review of Postgraduate Research Degrees, please seek advice if required from the Doctoral School.
The purpose of APR is to ensure that, in line with QAA requirements, all programmes are routinely monitored to evaluate their continuing effectiveness and currency, with actions being identified to remedy any shortcomings.
APR collects together information provided by external examiners, student feedback, and management information provided by the Quality Assurance Team. It provides a concise snapshot of one year in the delivery of a programme, and contributes both to annual overview taken by the department and School, and to comprehensive review of the programme through CDR. APR is conducted for all programmes, both taught and research.
B.4.2 Process for review of taught programmes
Following discussion at various committees during the session, and approval by the Teaching, Learning and Student Outcomes Committee, revised procedures for the annual review of taught programmes were introduced for the review of the 2010/11 session. Subsequent end of session reviews have taken place and minor adjustments have been made each year. Please see the guidance notes for full details.
Reporting takes place at programme and departmental levels by means of electronically submitted proformas. At programme level, the programme convenor reflects on input from module convenors, students and external stakeholders (including external examiners) as well as programme data supplied by the Planning Department before providing his/her own overview of the academic health of the programme. The programme convenor must confirm that the programme specification either remains appropriate for the forthcoming academic session or is being revised in line with the School’s amendment procedures. There is also the opportunity to identify and offer for broader dissemination examples of good practice arising from the operation of the programme. Targeted action plans are formulated to address any issues.
Programme level reports are then referred to the Head of Department who produces his/her own report to confirm that satisfactory programme reports have been received and considered at departmental meetings and that feedback on matters of concern raised and associated actions has been passed to students/examiners as appropriate. The departmental report also confirms that examples of good practice have been disseminated and that programme specifications are accurate or are being amended. Again, action planning is used to address any issues.
The cycle is completed when departmental reports are considered at a departmental meeting. This stage also asks that Head of Department highlight any matters relating to the implementation of good practice or matters of concern which the TELSOC might assist with. Finally, the department report includes a summary of themes arising from external examiner reports. Department reports are then considered by the Teaching, Learning and Student Outcomes Committee.
Please see the review cycle diagram for details of the operation of the cycle for the review of taught programmes.
The School’s policy is to conduct periodic reviews of all its programmes. Each programme will normally be reviewed once every six years. Programmes will be reviewed either along with the programmes offered in the same Department or, in the case of programmes which do not belong to any one Department, alongside other such programmes which are similar in subject area. This process is untaken as part of the Comprehensive Departmental Review (CDR).
Comprehensive Departmental Reviews come under the remit of the TELSOC, and are supported by the Quality Assurance Team and the Student and Acdemic Support Team. Panels report to TELSOC following reviews, and make recommendations to the Department and to central departments or committees of the School as appropriate.
Portfolio Review is the process of reviewing the modules offered by a Department (or other academic unit) and deciding which of them is viable to continue to run in the academic session after next. It is the first stage of the planning process, and feeds in to the approval of new programmes, decisions regarding staff and student recruitment, and other aspects of planning. The process by which Portfolio Review is carried out was approved by Governing Body in April 2010. The procedure is co-ordinated by the QA team, and supported by Department offices.
Portfolio Review is carried out at the very beginning of the academic year, at the same time as Annual Programme Review. Heads of Department receive a list of all the modules offered by their Department which in the previous session either (a) did not meet their quota of students (the quota agreed for 2009/10 was eight students) or (b) were on the books but did not run. The spreadsheet includes historical recruitment figures for modules which have been eligible for portfolio review previously, and comments made at that time by the Department, for information and updating. They are asked to put each of these modules into one of four categories for action in the coming year, and return the list to the QA team by the end of September. These actions are then considered and approved in Term 1, and subsequently implemented by the relevant committees, so that the status of each module is clear in plenty of time for online module registration to open between Terms 2 and 3.
Where a department approves categorisation of a module as (4) for the first time, their recommendation will be forwarded to Academic Board for confirmation. Thereafter the module will remain in category 4 until the next Comprehensive Departmental Review, when panels (which include external subject specialists) will be asked to make a recommendation as to its continuation. CDR panels will receive all annual reviews for the period and check that planned actions have been carried through.