How was the framework developed?

Research underpinning the framework’s design
This framework for the evaluation of teaching achievement during academic promotion marks the midpoint of an ongoing study commissioned by the Royal Academy of Engineering. The study is being undertaken in two phases, the first of which is now completed, as described below.
Phase 1 (January – August 2015)
Phase 1 of the study drew on international knowledge and best practice to develop a provisional evidence-informed framework for the evaluation of teaching achievement in higher education. It was informed by four evidence sources:
  • benchmarking of the evidence of teaching achievement requested during academic promotion to full professorship (or equivalent) at the world’s top-ranked universities, involving desktop examination of promotion criteria and consultations with and feedback from representatives of many of the universities concerned
  • interviews with international experts in the field of pedagogical competence, the measurement of teaching achievement and university promotion procedures
  • reviews of the research literature on existing and proposed measures of teaching achievement, from within and outside higher education
  • analysis and review of international good practice in the evidencing and evaluation of university teaching achievement, involving interviews with those engaged in designing, implementing and using these systems. The review included the compilation of a number of illustrative case studies of good practice at the institutional level, including Chalmers University of Technology, the National University of Singapore, and the University of Edinburgh
On completion of the first phase of work, the draft framework was reviewed by a group of carefully selected experts in teaching and learning to gather their feedback and guidance. A number of adjustments to the framework’s format, content and tone were made in response to this feedback.
Show more
Phase 2 (from September 2015)
Phase 2 of the study seeks to evaluate how well the framework works in practice. It will capture feedback on its design, approach and impact from a university-wide perspective and thus enable iterative improvements to be made. Participating universities have been selected from the group of institutions, identified in Phase 1 of the work, that are currently engaged in internal discussions about the potential for reshaping their promotion process.
Around 15 universities from across the world will be engaged in this phase of work, providing institutional feedback about the applicability of the framework and the potential challenges likely to be faced during its implementation in practice. A subset of these universities will be piloting the framework within their promotion systems. More information on these university partners is provided here.
Participating universities have been asked to address the following four questions:
  • Does the framework have the potential to improve how teaching and learning is evaluated and recognised at your institution?
  • Could the design/approach of the framework be improved?
  • What advice should be given to other universities wishing to implement the framework within their own promotion processes?
  • What potential exists to establish an international standard for university teaching achievements, allowing them to be ‘portable’, recognised across institutions across the world in a similar way to research achievements?
The final report from the study will be published by the Royal Academy of Engineering in mid 2017, providing the updated framework, the research underpinning its development and guidance for its implementation in practice.
Show more
Principles guiding the framework’s design
The study overall is focused on developing a new framework for evaluating teaching achievement during academic promotion which both defines a minimum threshold of achievement that increases with career progression, and offers opportunity for the recognition of additional achievement beyond this minimum. Outcomes of the previous and current study (to date) have also made clear that the framework needs to address a number of additional priorities and challenges. On this basis, the framework was designed to:
  • Be flexible, portable and commensurable with research criteria
    Across the academic community and within each discipline, the core measures of research achievement are well understood and recognised. Beyond standard measures of scientific excellence, these promotion systems typically have the flexibility to recognise different types of research contribution – for example, to industrial impact or to prestigious publications – and academics would not be required to contribute equally to both domains to meet the promotion criteria. In an equivalent way, the framework for evaluating teaching achievement must offer flexibility and transparency.

    At the same time, the framework should be designed to offer a clear set of definitions and criteria that are not bounded by disciplinary, institutional or national conventions, maximising the opportunities for achievements to be transferable between institutions. In this way, teaching achievements would be ‘portable’; recognised by other universities in an equivalent manner to research achievements.
  • Apply to ‘teaching and research’ academics as well as education-focused academics
    Although many universities across the world are reconsidering how they recognise and reward educational contributions, in the majority of cases, these discussions are focused exclusively on the institution’s education-focused career pathway – for those in specialist teaching and educational roles.

    At many institutions, the consideration of teaching achievement during the promotion of conventional teaching and research (T&R) academic is confined to ensuring that they meet a minimum threshold of acceptability. In many cases, progressive improvements in educational performance and impact beyond this threshold are not recognised or rewarded by their institutions. In other words, while progressive improvements in research achievement are a fundamental requirement for advancement up each rung of the T&R career ladder, equivalent improvements in teaching achievement are often not expected.

    The framework is designed to evaluate and recognise teaching achievement amongst both education-focused and T&R staff. It allows universities, should they wish, to embed a progressive increase in the minimum threshold for acceptable teaching. The framework also accommodates a range of levels of teaching achievement that mark advancement beyond this minimum threshold, allowing T&R academics the opportunity to place a greater weight on these contributions in their promotion cases.
  • Support professional development in teaching and learning
    In contrast to research achievement – where academics engage in a continuous and rigorous process of peer review via routes such as journal publications and research grant capture – ongoing evaluation of individual teaching achievement is not an accepted feature of the academic culture. The design of the framework must be sensitive to this environment, and the process for evidencing and evaluating teaching achievement should not be overly burdensome for candidates or university promotion committees. It should also inform the design of their continuing professional development in teaching and learning, allowing academics to structure their progress towards each step on the university promotion pathway.
  • Recognise contribution to educational practice as well as educational scholarship
    The emergence of teaching and learning career pathways in universities across the world has brought a reliance on educational scholarship (or pedagogical research) as often the primary criterion for advancement to more senior levels, particularly at research-led institutions.

    There is no doubt that contribution to pedagogical knowledge is one important marker of achievement, particularly for those mainstreaming in teaching and learning at more senior levels. However, scholarship-driven rewards processes often fail to recognise contributions made by academics to improving and supporting the teaching and learning environment, despite the wide-reaching impact that such contributions can have within and beyond the candidate’s institution. Examples include driving systemic curricular change or leading institutional teaching and learning strategy development/review.

    The framework should therefore have the facility to support academic progression on the basis of contributions to educational practice, both in the candidate’s institution and more broadly across the university sector, as well as on the basis of contributions to educational scholarship. It should also recognise contributions to nurturing a collegial and supportive educational culture across teaching staff within the candidate’s group or discipline; an environment shown to support the development of an effective and coherent programme of education (Fisher et al., 2003; Graham, 2012).
    Fisher, P. D., Fairweather, J. S., & Amey, M. (2003). Systemic reform in undergraduate engineering education: the role of collective responsibility. International Journal of Engineering Education, 19(6), 768–776. [link]

    Graham, R. (2012). Achieving excellence in engineering education: the ingredients of successful change. London: The Royal Academy of Engineering. [link]
  • Provide clarity about the forms of evidence that can support a promotion case
    Institutional promotion guidelines often ask candidates to provide, for example, “evidence of how you have improved student learning” or “evidence of innovations in pedagogy”. However, limited advice is typically offered about the forms of evidence that would be considered suitable or how such information could be collected and presented. As a result, many candidates rely heavily on single sources of evidence, typically student evaluation scores. Indeed, the evaluation of promotion guidelines among the world’s leading universities, conducted during Phase 1 of this study, revealed that many institutions do not provide any clear guidance about the forms of evidence that would support the educational elements of a promotion case.

    In addition, there is often a lack of distinction between teaching-based promotion criteria (the characteristics of teaching achievement that the institution would look for in a successful candidate) and teaching-based evidence (the qualitative and quantitative data that could/should be provided to demonstrate the candidate’s achievement of the criteria). Indeed, promotion guidelines at many universities appear to confuse the two, listing sources of evidence (such as peer-reviewed educational publications) within the promotion criteria or listing promotion criteria (such as “demonstrating that good conditions for student learning have been established”) as a suggested form of evidence to include in a promotion case. This lack of clarity appears to add further confusion to the process of identifying and collecting evidence to support a promotion case.

    The framework should be clear about the types of evidence that promotion candidates could use to demonstrate teaching achievement, with guidance on how this information can be gathered in practice.