“Quality, whilst not easy to define, is mainly a result of the interaction between teachers, students and the institutional learning environment. Quality assurance should ensure a learning environment in which the content of programmes, learning opportunities and facilities are fit for purpose. At the heart of all quality assurance activities are the twin purposes of accountability and enhancement. Taken together, these create trust in the higher education institution’s performance. A successfully implemented quality assurance system will provide information to assure the higher education institution and the public of the quality of the higher education institution’s activities (accountability) as well as provide advice and recommendations on how it might improve what it is doing (enhancement). Quality assurance and quality enhancement are thus inter-related. They can support the development of a quality culture that is embraced by all: from the students and academic staff to the institutional leadership and management.”
(European Standards and Guidelines, 2015: 7)
The PPR Policy & Procedures full document is available here [PDF]. Below are the document's main sections, which can be accessed and referred to as required (each section also includes the full section in PDF where relevant).
View full section [PDF]
This document defines the principles, purposes and procedures that underpin the University of Malta’s academic programme reviews and monitoring.
The following four principles for quality assurance (QA) in the European Higher Education Area (ESG, 2015) are considered here:
The Periodic Programme Review (PPR) is an integral component of the University of Malta (UM)’s Internal QA. It is a rolling system of peer review, in which all academic programmes of UM are reviewed on a five-to-six-year cycle. This policy and procedures do not preclude the Rector from calling for review of an academic programme for other purposes not covered here.
Institutional responsibility and pride inspire us to evaluate our academic programmes. This will lead to critical reflection of the relevance, appropriateness and utility of what we teach in relation to academic progress, the changing needs of the local economy as well as the global context in which the University exists. QA also provides a platform for continuous enhancement through both internal and external stakeholders’ feedback which in turn contributes to improvements in content, learning outcomes, means of delivery, as well as mode/s of assessment.
The University of Malta’s PPR process draws on Malta’s National Quality Assurance Framework for Further and Higher Education (NCFHE, 2015) and adheres to Internal QA standards as well as the principle of “ongoing monitoring and periodic review of programmes to ensure objectives are met and respond to the needs of the students and society” (NCFHE, 2015). Underpinning the PPR process and a prerequisite for success is continuous enhancement as embodied by the quality cycle.
The aim of the PPR is to monitor the quality and standards of the provision of the University’s educational programmes, qualifications and courses. It provides the basis for ongoing quality enhancement to assist in achieving excellence in learning, teaching and research at UM.
It is intended that PPRs are carried out in a spirit of open, collegial discussion through a developmental approach with the overarching aim of continuous enhancement. They are not auditing of past performance, but rather opportunities for transparent professional dialogue and meaningful reflection and to steer forward planning to ensure that our programmes are relevant, current and effective in providing a high-quality learning experience for our students and to
equip them for success as graduates.
The objectives of the PPR are to:
All FICS are reviewed systematically on a five-to-six-year cycle. A schedule of reviews is agreed by the Quality Assurance Committee (QAC) and will be published on UM’s website.
FICS with programmes accredited by foreign professional, statutory and regulatory bodies are encouraged to explore appropriate ways of aligning the external activities with the PPR process. This might include the use of common documentation or joint processes which meet the needs of both the PPR and the external accreditation and evaluation.
The PPR schedule is available here [PDF].
View full section [PDF]
The key stages of the Periodic Programme Review (PPR) process are outlined below with indicative timeframes.
View full section [PDF]
The Pro-Rector for Quality Assurance will appoint an IQR panel which will normally comprise:
The Pro-Rector for Quality Assurance or their delegate will act as Chair. The primary role of this panel is to drive the PPR process and ensure that the quality and standards of all UM programmes are maintained. In liaison with the Pro-Rector for Quality Assurance, the IQR panel shall organise an initial meeting with the FICS PPR Committee to discuss and clarify the PPR process as well as provide guidance on how the information will be collected, analysed,
interpreted and used for the purpose of continuous enhancement.
The role of the IQR panel will also include:
View full section [PDF]
Upon being informed about the upcoming PPR, FICS Boards will need to establish a PPR Committee. This may be an already established Board of Studies. The decision on the composition of the PPR Committee shall be driven by practical considerations, such as the
number of programmes for review, the size of the FICS, etc. The PPR Committee will be chaired by a person who shall be tasked with driving this process in a timely manner. The Chair of the PPR Committee will act as point of reference and coordinate the collection of data and material. In collaboration with the other members of the PPR Committee, the Chair shall prepare the Self- Evaluation Document (SED) with input from colleagues. Students should also be consulted and the Faculty Board/Board of Studies should be given the opportunity to comment on the SED prior to submission. The PPR Committee should ideally have a secretary.
The PPR Committee shall be responsible for:
View full section [PDF]
The Stakeholders’ Committee (SC) will normally comprise:
The external experts in (c) and the student representatives in (d) will be nominated by the PPR Committee and endorsed by the Dean/Director in liaison with the Pro-Rector for Quality Assurance. The SC plays a vital role in assisting FICS to identify strengths and key issues which need to be addressed and enhanced.
The role of the SC includes:
View full section [PDF]
Each programme has documented evidence of its focus, intended competency achievements, and learning outcomes. This documentation – often fragmented in the Stage One/Stage Two proposal forms which are submitted to the Programme Validation Committee (PVC) with respect to any new and substantially revised programmes of study, the prospectus documents, and other informational materials – present an idealised vision of what the programme intends to achieve. A synthesis of this documentation provides a starting point for any gap analyses and actions that may be undertaken during the determination of the merit and worth of the programme.
It is the practice of this University to have a Board of Studies (BoS) for each and every programme. This BoS oversees the day-to-day management of the programme including distribution of study-units, delivery of instruction, results, etc. At the end of each academic year the BoS will carry out an Annual Programme Review (APR), to formalise evaluation of and feedback about the programme for the past year. This material, together with the minutes, discussions, and recommendations of the BoS could be used by the PPR Committee as a brief historical background of the programme highlighting achievements and/or issues and concerns.
A summary of the issues arising from External Examiners’ reports and Boards of Examiners’ meetings provides information about accomplishments, problem identification and assessment from both an internal and external perspective. Any action identified as a result of such reports and comments should also be outlined here.
Students are a major stakeholder of any academic programme. Their reactions, taken judiciously within context, can offer a number of useful pointers, indicating both strengths and areas for development. A summary of the issues raised through the student feedback exercises (both at study-unit level and prgramme level), together with issues raised by student representatives on the BoS and the FICS Board, represent further evidence of programme operation. The PPR Committee should consider whether the feedback was satisfactory, what strengths and areas for development were identified, whether any issues for concern were raised, and what action has been or should be taken as a result of feedback. Focus groups with current students can enrich knowledge about programme implementation. Furthermore, when possible, representative examples of student feedback could strengthen the documentation of student reactions.
Student beneficiaries of our programmes are often a heterogeneous group. Some attend a programme of study-units as their main area of study while others take a subset of the study-units as their main ordinary area, their subsidiary area or even as optional units. When collecting information from students, it is advisable to keep these motivational differences in mind.
An overall summary of the applicants’ profile and numbers at entry point provides evidence of the target audience of the programme. Details such as age, gender, full-time/part-time status and students declaring a disability can point to gaps that require remedy when determining worth. Other indicators of student behaviour in the programme include a summary of progression and attrition rates, based upon the percentage of those enrolled who are subsequently successful at each stage of the programme.
Alumni of the programme contribute a long-term perspective of the merit and worth of a programme. What kind of employment do graduates of the programme enter upon leaving the programme? What career paths have they taken and how long does it take alumni to get promotions? Do the knowledge and skills acquired during the degree match the jobs they obtain? Do any students proceed to create their own enterprise? What proportion of students proceed to read for further studies and what proportion is unemployed over time? How does this compare with unemployment rates for graduates of other programmes? Is there any evidence of a mismatch in what is being taught to students and the demands and needs of the labour market and, if yes, what action is planned to address such a discrepancy?
Honours undergraduate programmes normally require a dissertation in partial fulfilment of the degree requirements. The dissertations themselves can be evidence of rigour, academic stature and quality. A brief overview of the topics covered and their respective reports can provide ample evidence of student work in a particular programme. This is even more the case for taught postgraduate programmes which also include a dissertation. One might consult the dissertations with their respective reports (especially those of the external examiners) as further evidence of the type and quality of work generated by the programme.
Academics, whether resident or visiting, can also provide their perception and experience of the programme. They can speak of their own involvement with students, the support they receive from the University and the FICS involved. They can address the quality of interaction among faculty and students.
External stakeholders such as professional associations (where relevant), beneficiaries and employers offer another set of perspectives and reactions to the programme. Although one
should not expect employers to make a clear distinction between education and training, they contribute pivotal information about the programme. Similar caveats need to be present when collecting information from beneficiaries. The experience and perceptions of these groups could be collected through reports or focus groups as appropriate.
The data collection exercise provides a collage of perceptions, attitudes and experiences of the programme’s stakeholders. It is the PPR Committee’s responsibility to capture the richness of nuances and reactions presented together with the motivations expressed by the different stakeholders. The main goal is to analyse the incoming information, interpret it and use it in meaningful ways to address any identified gaps by proposing actions to be taken to steer
continuous enhancement.
It is highly probable that, in collecting such diverse information, the PPR Committee encounters conflicting information about the programme arising from the different interests involved. The
PPR Committee, while being loyal to the diversity of voices, should exercise its judgement in the presentation of the overall academic picture of the programme.
View full section [PDF]
The key document produced by the PPR Committee is the Self-Evaluation Document (SED). The preparation of the SED is a very important part of the PPR and should include ‘core’ information
presented in a structured way. This should be analytical and evaluative rather than descriptive. It is also the key document through which the FICS conveys a snapshot about itself. The preparation of the SED also serves as a starting point for critical reflection by the FICS about the way it is organised and managed, the mechanisms used to evaluate its activities and how these are sustaining continuous enhancement. It is an opportunity for the FICS to trigger an open professional dialogue with internal and external stakeholders about what is working well, less well, and what might be done to address areas for development.
It is essential that there is full consultation with all members of the FICS during the preparation of the SED. The draft SED should be submitted to all academic staff members for comment and fully
discussed at a meeting. Students should also be given the opportunity to comment on the SED – this can often best be done either through a special meeting or focus groups in consultation with the FICS student representatives and/or student/staff liaison mechanisms.
The document should aim to be around 12-15 pages and should not exceed 20 pages (excluding appendices). A full SED template is available here [Word], and the Assessment Brief template is available here [Word].
View full section [PDF]
The Internal Quality Review (IQR) panel, in liaison with Pro-Rector for Quality Assurance, schedules an initial meeting with FICS Dean/Director and the PPR Committee. The aim of this initial meeting is to discuss, clarify any queries and plan ahead for the PPR process. It will also serve to introduce the IQR panel with the FICS Dean/Director and the PPR Committee.
At this stage, a mutually suitable date for the Quality Collaboration Visit will be agreed upon during this initial meeting. The Quality Collaboration Visit is not normally held at the beginning of Semester One (October) or during recess period, revision or placements and exam periods.
View full section [PDF]
The Quality Collaboration Visit is normally held over one day. Exceptions to this may be made when a large number of programmes are being considered and this will be discussed and agreed upon with FICS. The Quality Collaboration Visit is intended to be a positive and valuable process for the FICS. It aims to recognise and commend good practice, and support the enhancement of provision and the student learning experience. Colleagues will be encouraged to discuss the operation of their FICS, reflect on issues and challenges, and highlight examples of good practice worthy of dissemination across the University. Colleagues are warmly encouraged to contribute as fully and openly as possible in meetings. Aspects evidenced as routinely going well may not be discussed during this visit but may feature in the final PPR report. The Internal Quality Review (IQR) panel will focus on innovative activities and areas of interest identified in the SED as the key themes. Other discussion topics of interest may emerge during the course of the visit. Practical arrangements for the visit may be found here [PDF].
As part of this visit the IQR panel meets with three separate panels:
Members of panels (b) and (c) should not normally include members of panel (a).
The panels of students will be conducted via parallel sessions to ensure that the views of each level of study are represented and captured. The IQR panel should be mindful that the FICS staff/students may feel apprehensive about the visit. Efforts should be made to ensure that those meeting with the IQR panel are made to feel as comfortable as possible. The IQR panel Chair should ensure that the meetings focus on professional dialogue, are conversational and that all the participants are given an opportunity to share their views.
To conclude the visit, the IQR panel meets the Dean/Director and PPR Committee to clarify any points and to formally thank them for their participation the final meeting will also be an opportunity for the IQR panel to reflect on commendations and recommendations.
View full section [PDF]
The Stakeholders’ Meeting (SM) is to be organised and hosted in a timely manner by the Internal Quality Review (IQR) panel in liaison with the FICS undergoing the PPR process and with the Pro-Rector for Quality Assurance. This meeting should include all members of the Stakeholders’ Committee (SC) and the IQR panel and will be chaired by the Pro-Rector for Quality Assurance or their delegate. The SM may be held in person, online or in hybrid form, although the first option
is preferable. A 2-hour session is generally recommended for the SM.
The IQR panel will share the agenda, together with the finalised SED, with all participants prior to the meeting. If the PPR concerns multiple programmes, the agenda should allow for an equal period of time to discuss each programme. Time management is critical, and the Chair should ensure that each and every participant has the time and opportunity to offer their observations.
By way of introduction, the FICS Dean/Director may make a short presentation (not more than five slides if using PowerPoint) to briefly set the backdrop for the meeting and present the SED. This should not take more than fifteen minutes, since the main purpose of this meeting is to receive feedback from the stakeholders through a healthy discussion. The QSU shall take minutes of the proceedings, which will then be shared with the PPR Committee Chair. A template for a typical agenda may be found here [Word].
During the meeting, members of the SC are encouraged to provide three main forms of feedback:
Members of the SC may base their observations both on their personal experience stemming from their relationship with the FICS, as well as with regard to the content of the SED prepared by the FICS.
View full section [PDF]
The final PPR report will be prepared by the IQR panel and will include the summary of findings and outcomes emerging from the Self-Evaluation Document (SED), the Quality Collaboration Visit and the Stakeholders’ Meeting (SM). The PPR report is submitted to the PPR Committee Chair to be discussed internally at FICS Board level, provide reactions and clarifications in the report’s final section and prepare a plan of action as to how it shall be addressing the recommendations in the PPR report. Should the FICS not agree with any of the tabled recommendations, a clear justification is required by its PPR Committee.
The PPR report, with the FICS reactions and the plan of action, is submitted to the IQR panel, which may revert to FICS for further discussions or clarifications if required. The finalised PPR report is submitted to the Pro-Rector for Quality Assurance and the Pro-Rector for Academic Affairs for their consideration, and then presented to the Senate.
FICS are responsible for the implementation of action plans arising from the PPR. However, in cases where urgent action is required, the Pro-Rector for Quality Assurance and/or the Pro- Rector for Academic Affairs, in consultation with the Rector, may reserve the right to execute and/or monitor such implementation.
The PPR report will include the following:
1. The summary of the findings emerging from the SED, the Quality Collaboration Visit and the Stakeholders’ Meeting will be organised as follows:
Commendations: these are the areas identified as good practice or strengths of the programme. Good practice may be defined as effective practice beyond that defined by regulations or policy such as any innovations that enhance learning and teaching, the student experience in general or any practice which was developed to meet a particular need. Strengths of a programme may be identified as any particular positive features of that programme and that may be sustaining the UM’s Strategic Plan 2020-2025.
Recommendations: clear recommendations for enhancement of the programme under review expressed in terms of actions to be taken by FICS against a realistic timeframe.
Conditions: where required, measures and conditions may be specified to fully support quality and standards. Such conditions must be addressed through immediate action and/or in a short timeframe, as they may seriously affect the quality of the programme as well as
overall student experience and/or wellbeing.
2. Plan of action
The FICS will be asked to submit an action plan to outline intended actions and timescales to address the recommendations in the PPR report. The action plan should be discussed with the FICS Board, members of staff, the BoS and where possible, the students prior to
submission. The action plan should include a statement on the steps taken to share with staff and where possible with students.
The IQR panel will review the plan to ensure that the recommendations have been adequately addressed, and that staff and students received feedback on the outcomes of the review and were consulted on the production of the plan.
The plan of action for the areas of enhancement identified through the PPR report must feature in the FICS Annual Programme Review and will be followed up by the Pro-Rector for Quality Assurance and the Pro-Rector for Academic Affairs as deemed appropriate.
A sample action plan may be viewed here [PDF]