Evaluations – quality in education

2024

  • Evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10): About the master's thesis

    Read the report (in Norwegian | pdf)


    Summary

    NOKUT is conducting an evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10), in 2022–2024. As part of the evaluation, NOKUT conducted surveys among students, candidates, campus-based teacher educators and initial schoolteachers having a role as school-based teacher educators. This report compiles and describes the results of the surveys about the master's thesis.

    Most students understand the purpose of the master's thesis
    The majority of students understand why they are required to write a master's thesis. However, some students who understand the purpose of the thesis believe that it would be more beneficial to prioritize learning other skills during their education.

    Well-prepared
    The vast majority of students feel well-prepared for the work on the master's thesis, partly due to introductions to research ethics, methodology courses, and supervision. Many of the teacher educators who supervise in master's thesis work also indicated that they are involved in various preparatory activities.

    The learning activities that most fifth-year students reported having experience with are "receiving training/guidance in working with problem statements and project design" (90 percent). The fewest ticked off "receiving guidance from practice teachers during the preparatory phase" (25 percent).

    In an open-ended text field, many fifth-year students chose to highlight good guidance from the campus-based teacher educators as important in their preparations. Most are also positive about the guidance they have received.

    About one-third of the fifth-year students indicated "To a great extent" or "To a very great extent" that they have "participated in research projects" and have "carried out or participated in research or development projects in the classroom or school." A slight majority of the teacher educators stated that they "...facilitate students' participation in research projects."

    Choice of topic and problem statement
    The vast majority of fifth-year students reported having chosen the topic and problem statement for their master's thesis themselves. A significant minority reported having chosen in consultation with a teacher educator or in consultation with other students. A total of ten percent reported having received suggestions at master's fairs, seminars, etc.

    Almost all teacher educators stated that students largely choose their own topic and problem statement, while half replied that students largely need suggestions.

    Half of the students stated that it was easy to choose a topic, problem statement, and method. A minority, one-third, stated that it was easy to get started with writing.

    Type of assignment
    Half of the students collect data from schools, while a quarter collect data outside of schools. Furthermore, a quarter conduct literature studies/document studies/text analyses. Among those who collect data from schools, the most common methods are surveys or interviews with teachers, but many reported classroom observations, surveys or interviews with students, or implementation of teaching plans. Most instructors stated that students can use existing data instead of collecting data themselves, but very few students do.

    Very few students collect data during their school placement periods or develop the thesis in collaboration with school-based teacher educators.

    Supervision
    A clear majority of students stated that the supervision of the master’s thesis is good and that data collection is going well.

    The vast majority of those who supervise in master's thesis work responded that they participate in supervision meetings, while half stated that they have received training to supervise master's theses. Most teacher educators who supervise master's thesis work do not feel that they supervise too many master's theses at the same time. Most believe that they supervise within subjects they are competent in and that it is rewarding to supervise.

    The biggest challenges
    In an open-ended text field, we asked students what the biggest challenges are. The most frequently mentioned are (own) lack of motivation and self-discipline. In a similar open-ended text field, teacher educators highlight three factors: students' academic skills, the fact that master's thesis work has too little space (time), and the challenge of finding enough and competent supervisors.

    The distribution of grades
    The grades on the master’s thesis are generally highest in early childhood education and English and lowest in mathematics.

    Candidates who wrote together with others receive significantly better grades than those who wrote alone, especially in mathematics and Norwegian. Furthermore, there is a very clear correlation between grades from higher education and the grade on the master's thesis. This is found both in descriptive statistics and in regression analyses.

    Benefits of working on the master's thesis
    The students and candidates respond relatively similarly to most of the questions both groups were asked, but the candidates are consistently somewhat more positive about the learning outcomes than the students.

    Both the candidates and students have mixed opinions on the statement "working on the master's thesis prepared me well for doing exploratory work in various fields in the school": About one-third indicated a low degree of agreement, one-third medium agreement, and one-third a high degree of agreement. We find the same picture on the statement "Working on the master's thesis gave me important insights and knowledge that has prepared me well for my work as a teacher." The two groups are more in agreement on the statement "Working on the master's thesis taught me how to read research and extract key information," where less than ten percent indicated a low degree of agreement. Many also agree that "Working on the master's thesis taught me to realize the importance of being professionally and pedagogically updated."

    On several questions about how candidates use the competence gained from working on the master's thesis in their job as a teacher, it appears that the competence is used to a limited extent. It is worth noting that most had been working for less than one year at the time of response.

    In the open-ended text fields, around two-thirds of both the students, candidates, and campus-based teacher educators described a positive learning outcome from the master's thesis. All groups emphasized, perhaps not surprisingly, an outcome consisting of competence in research and analytical skills, and with it the opportunity to further develop themselves in the teaching profession. In addition, many highlighted subject-specific depth. The approximately one-third who were more negative about the learning outcome emphasized that the learning outcome is not relevant to the teaching profession.

    The candidates' perceived benefit of working on the master's thesis has a statistically significant correlation with the master's subject the thesis is written in. Candidates with pedagogy and early childhood education have a significantly higher probability of being satisfied with the outcome than those with Norwegian. This is controlled for many other factors in regression analyses. Furthermore, there is a strong correlation between perceived benefit and the grade on the master's thesis.

    The students' perceived benefit of working on the master's thesis has a statistically significant (and positive) correlation with their own motivation and whether they have participated in research projects during their studies.

    Many other factors do not have a statistically significant correlation with the students' and candidates' perception of the benefit. This applies to, for example, gender, age, grades from upper secondary school, grades from higher education, and whether they wrote together or alone. This is controlled for many other factors in regression analyses.

    Author: Pål Bakken

    Read the report (in Norwegian | pdf)

  • Evaluation of the initial teacher education for grades 1–7 and 5–10: About the school placement periods

    Read the report (in Norwegian | pdf)


    Summary

    NOKUT is conducting an evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10), in 2022–2024. As part of the evaluation, NOKUT conducted surveys among five groups of respondents: students, candidates, campus-based teacher educators, initial schoolteachers having a role as school-based teacher educators and (primary) school leaders. This report compiles and describes the results of the surveys about the school placement periods.

    Overall, Generally Positive, but more Negative in Some Areas
    On most of the questions, there are far more respondents indicating a high degree of agreement than a low degree. This suggests that most respondents have a positive view of school placement, considering how the questions are formulated. On the three questions about the benefits of school placement, the respondent groups overwhelmingly respond positively. This includes whether the learning goals for school placement periods were achieved, if one gains valuable experience in teaching subjects, and if one gains valuable experience in the professional areas beyond instruction. Additionally, in free-text fields, many students express satisfaction with the school placement and find it educational.

    Well-prepared for School Placement
    The school-based teacher educators are well-prepared for the arrival of students, students are well-prepared for school placement, and the organization of school placement is perceived as good.

    Guidance from Campus-based and School-based Teacher Educators
    The overall picture is that guidance from school-based teacher educators is good, while follow-up from campus-based teacher educators could be better. Students, campus-based and school-based teacher educators are divided in their views on whether guidance from campus-based teacher educators and school-based teacher educators is coordinated.

    Inclusion in the Academic Community during School Placement
    The majority, both school-based teacher educators and students, believes that students are well included in the academic community at the (primary) school.

    Work Requirements during School Placement
    In each of the respondent groups, very few respondents mention (in free-text fields) simultaneous work requirements during school placement as problematic.

    Integration of School Placement with the Rest of the Education
    The respondent groups are divided in their views on whether the sequence and progression of teaching and school placement periods are well aligned. Most students believe this is not the case, while most school-based teacher educators believe it is. Candidates and campus-based teacher educators are somewhat more in agreement with the statement than students. However, the majority in all respondent groups agrees that the placement of school placement periods in the study program and school year facilitates academic development as a teacher.

    Collaboration on School Placement
    The respondent groups are fairly unanimous in their assessment of how well the collaboration between campus-based teacher educators and school-based teacher educators is: there are slightly more who believe this is not the case than the opposite. On a more overarching statement about the collaboration between universities and (primary) schools, the majority of both campus-based and school-based teacher educators believe this is largely true.

    Communication and Connection between Teaching and School Placement
    Campus-based teacher educators, school-based teacher educators and school leaders point out the need for more and better communication and a better connection between campus teaching and what happens during school placement.

    Length and Extent of School Placement Periods
    Within each respondent group, there are different opinions about whether the school placement periods, as a whole, are long enough. Approximately the same number of respondents largely agrees as disagrees. In the free-text fields of the surveys, it was common among all respondent groups to express views on longer school placement periods.

    School Placements are Relevant
    Students, candidates, and school-based teacher educators agree that the school placement periods are in stages that are relevant and in subjects that are relevant to the students.

    Students Achieve Learning Goals
    It is common for either campus-based teacher educators or school-based teacher educators to go through the learning goals for the school placement period with the student, but a significant minority feel that this does not happen. Most students and school-based teacher educators believe that students achieve the learning goals in school placement periods.

    Learning Outcomes
    All the respondent groups, especially school-based teacher educators, believe that school placement periods provide students with good experience in teaching subjects. Most within each respondent group (except candidates) also believe that school placement periods give students good experience in the work and responsibilities of the teaching profession beyond instruction.

    Attractive to be an Instructor, but too Little Time
    Many school-based teacher educators find it attractive to instruct students. At the same time, many feel they have too little time to take care of students. Most have a role as a school-based teacher educator between four and ten weeks a year, and most spend less than 16 hours in a typical week with GLU students. However, as many as 15 percent reported spending more than 33 hours in the role as a school-based teacher educator. In a free-text field, many commented that the role is time-consuming.

    Author: Pål Bakken

    Read the report (in Norwegian | pdf)

  • Evaluation of the initial teacher education for grades 1–7 and grades 5–10: Survey among initial teacher education students

    Read the report (in Norwegian | pdf)


    Summary

    The GLU Students
    The population consists of registered students in initial teacher education programmes in the autumn of 2022. In total, 3,409 individuals completed the survey, representing 30 percent of the population.

    Little Difference in Response Patterns among Subgroups
    Students from initial teacher education for grades 1–7 and grades 5–10 respond similarly, or nearly so, to most questions. There is also little difference in response patterns when comparing campus-based education with education that combines distance learning with some campus-based days. Additionally, there are only minor gender differences. While there is variation between institutions, the differences are not typically very significant.

    More experienced Students are more Negative than less experienced Ones
    Students who have progressed further in their studies respond more negatively to many statements compared to "younger" students. The most experienced students are more negative or critical about various aspects of their studies than the least experienced ones. Age plays a minor role, as similar differences are observed when examining students within specific age groups.

    Most Students thrive, little variation among Subgroups
    The majority of students, around two-thirds, reported that they thrive to a great or very great extent in their studies. Just under ten percent stated that they thrive to a small or no extent.

    Motivation for the Study could be higher, but most are motivated for a Teaching Job
    Almost all students reported being highly or very highly motivated when they started their studies. However, this dropped to 50 percent when asked about their current motivation. Despite this decline, the majority of students remain well motivated to work as teachers.

    Subjects and Subject Choices should better suit School Needs
    Regarding several statements about subject choices, breadth, and specialization, the assertion that "It should be possible to take more elective subjects" is the one most agreed upon (66 percent). Many also believe that "The subject combinations one can choose should correspond more to the needs of the school" and that "The content of the study subjects is too specialized compared to the needs of the school."

    Many understand why They should write a Master's Thesis, but Some are skeptical
    A majority (56 percent) partially understood, 21 percent clearly understood, while 23 percent answered no to the question "Do you understand why you should write a master's thesis?" There is little variation in responses based on the students' progress in their studies.

    Well prepared for working on the Master's Thesis
    Students received training in various skills relevant to research and thesis writing. Fifth-year students, in particular, received preparation for the master's thesis, including guidance in defining problems and project design, academic writing, and preparation phase guidance.

    Common to collect Data for the Master's Thesis from Schools
    The majority of fifth-year students collect their own data, most commonly from schools, through methods such as surveys or interviews with teachers.

    Diverse Views on the Outcome of working on the Master's Thesis
    Fifth-year students have mixed opinions on whether the master's thesis prepares them well for enquiry-based work in various fields of education. There is also a split on whether it provides essential insights and knowledge for teaching.

    Teaching: More Emphasis on Research and less on School Practice over time
    As students progress in their studies, they perceive a greater emphasis on research and less on the school practice in teaching. This shift is reflected in their views on teaching methods and the use of faculty research in instruction.

    Most Lecturers have Experience and Expertise from Research
    The majority of students believe that faculty members primarily derive their experience and expertise from research, with a minority suggesting it comes mainly from the schools. While many students agree that faculty members have sufficient relevant expertise to educate teachers, some feel that these educators lack experience and competence from schools.

    Program Structure and Coordination could be better
    Many students feel that there is little "good academic coherence between the course PEL (Pedagogical Education and Learning) and subject courses." They also express dissatisfaction with the coordination and progression of teaching and school placement periods.

    Positive Perception of the Relevance of School Placement
    Most students find their school placement relevant in terms of grade levels and subjects. They also appreciate guidance from both campus and school-based teacher educators during these periods.

    Follow-up/Guidance from Campus and School-based Teacher Educators during School Placement
    Most students experience school placement-related guidance/support from campus-based teacher educators and simultaneous guidance from both campus and school-based teacher educators. However, a significant minority feels they receive little support from the campus-based teacher educators at the university.

    Well-organized School Placement
    The majority of students agree that the final school placement period is well-organized, receiving the necessary information beforehand, and that the school-based teacher educators are well-prepared for their arrival.

    Length of School Placement
    In response to the question about whether the school placement could be improved, many students suggest longer school placement periods, contrary to their earlier agreement that the periods are long enough. This suggests that the issue of school placement length is more significant for those who are dissatisfied.

    School Placement provides valuable Educational Experience
    Most students agree that the school placement provide valuable experience in teaching their subjects and in the broader responsibilities of the teaching profession. Almost all students feel they have achieved the learning objectives for their last school placement period.  

    Authors: Pål Bakken and Erlend Langørgen

    Read the report (in Norwegian | pdf)

  • Evaluation of the initial teacher education for grades 1–7 and 5–10: Survey among campus-based teacher educators

    Read the report (in Norwegian | pdf)


    Summary

    NOKUT is conducting an evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10). The evaluation is scheduled to be completed in the autumn of 2024. The report presents results from a survey among campus-based teacher educators. 

    Educational Competence of the Campus-Based Teacher Educators
    About 60 percent of the respondents in the survey have disciplinary education, while about 60 percent have some form of pedagogical or teacher education. Many have multiple types of education, so the proportion that only has disciplinary education is around 30 percent, and the proportion that only has initial teacher education is around 10 percent.

    Main Emphasis of Experience and Competence from Higher Education
    Around 70 percent stated that they have a large or very large part of their experience and competence from teaching in higher education, 42 percent from research, and 29 percent from primary schools.Distributed by main position, very few professors and associate professors have the majority of their experience from primary schools, while this is much more common among assistant professors and university college teachers.

    The Relationship between Research and Teaching
    The majority of the respondents conduct research on issues directly related to the school system and use their own research in teaching. Almost all use other research in teaching and regularly update the academic content in teaching to reflect new research. Furthermore, the vast majority use teaching methods and approaches rooted in research on student learning and keep themselves updated on research about student learning.

    Academic Integration
    A third of the respondents answered 'No' to the question about whether they have an overview of the students' workload in subjects other than the ones they teach themselves. Collaboration with colleagues in their own field is very common. Much collaboration with colleagues in other fields or with teachers in primary schools is uncommon, but a good number of respondents collaborate to some extent with these groups.

    Positive and Negative Aspects of School Placement
    Most respondents have high confidence that the school-based teacher educators provide students with good guidance and that they are well prepared for the practice students. Furthermore, most believe that the collaboration with the schools is good. Most respondents also believe that students gain good experience in teaching their study subjects and in the professional areas of teaching beyond instruction. Despite the predominantly positive responses to these statements, a minority believe that the school placement periods are not long enough and that there is not enough collaboration between school-based teacher educators and campus-based teacher educators. According to the respondents, there is room for improvement.

    In a diligently filled free-text field with the question 'What is most important to improve school placement?', many wrote about the collaboration between universities and the practice field, stating that this is an area that can be improved. There is a need for more meeting places, better communication, and the development of a common understanding of roles where school-based and campus-based teacher educators respect each other's contributions to the education of primary school teachers. Many respondents expressed a need for longer and/or more school placement periods.

    Campus-based teacher educators in School Placement
    Most of the respondents who supervise students during school placement go through the learning objectives together with the students, feel that their own and the school-based teacher educators' guidance is coordinated, and give students academic assignments related to their school placement.

    Interdisciplinarity
    The respondents were given three statements about interdisciplinarity: one about interdisciplinary themes in the curriculum being integrated into large parts of the education, one about emphasis being placed on developing competence to work interdisciplinary and multidisciplinary in schools, and one about emphasis being placed on theme-based teaching. About a quarter stated that they did not know anything about this. Among the remaining respondents, about a quarter stated 'To a large extent' or 'To a very large extent' on the three statements, while slightly more stated 'To none/very little extent' or 'To a small extent', and about half 'To some extent'. The respondents are therefore quite divided in their perception here.

    Structure of the Study Program
    The vast majority of the respondents do not agree that there is good academic coherence between PEL (Primary Education and Teaching) and the study subjects. Approximately the same number of respondents agree with the statement that the sequence and progression in teaching and school placement periods are well adapted to each other, as there are respondents who disagree with the statement.

    Students' Work on the Master's Thesis
    The vast majority of the respondents stated that students link the master's thesis to their own practice and that students are offered to participate in research groups/projects. About half stated that students can submit assignments in various formats (artistic product, research article, etc.), which means that about half stated that this is not the case.

    Supervision of Master's Thesis
    The vast majority of those who supervise in master's thesis work do not think they supervise too many master's theses simultaneously. The vast majority agree that they supervise within subjects they are competent in, that it is rewarding to supervise, and that they can develop their competence. The overall picture this provides is that it is positive to supervise students' work with their master's thesis.

    Challenges with the Master's Thesis
    In the free-text field, many respondents stated that students' academic skills and competence are at too low a level to write good master's thesis. The time aspect was also problematized – it was pointed out that the master’s thesis is given a smaller part in the teachers’ education, and that there is not enough time provided to write high-quality master's theses. Moreover, it is a challenge that many students work a lot alongside their studies. Several also highlighted the students' lack of motivation, and the perceived and actual relevance of the master's thesis for the teaching profession as challenging.

    From the perspective of educational institutions, many pointed out that it is challenging to find enough and competent supervisors, since there are many master's theses written annually. Furthermore, it was considered problematic that there is no consensus on what a master's thesis in GLU should entail – regarding its purpose, content, format, and grading.

    How the Work with the Master's Thesis qualifies the Students
    The majority believe that the work strengthens students' academic competence within the themes they choose, as well as their methodological and analytical skills. The skills students acquire will enable them to research their own practice and drive professional development, as several emphasize.

    However, about a third of the respondents were critical of the utility of a master's thesis. Some pointed out that the usefulness of the master's thesis depends on the student's competence and ambitions, and the relevance of the theme and/or methodology for the teaching profession. Furthermore, some disagreed that the work produces better-prepared candidates. It is pointed out that there are other measures that would make students better qualified for the everyday life as a teacher than a master's thesis would.

    Students' Final Competence
    Overall, the campus-based teacher educators paint a fairly positive picture of the students' final competence when they answered three statements about the extent to which students are well prepared for their first job as a teacher, whether they have necessary knowledge, skills, and competence, and whether they have a good basis for further development of their own practice and the teaching profession.

    Authors: Pål Bakken and Ingebjørg Flaata Bjaaland

    Read the report (in Norwegian | pdf)

  • Evaluation of the initial teacher education for grades 1–7 and 5–10: Survey among school campus-based teacher educators

    Read the report (in Norwegian | pdf)


    Summary

    NOKUT is conducting an evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10). The evaluation is scheduled to be completed in the autumn of 2024. The report presents results from a survey among school campus-based teacher educators, working in primary schools with GLU students.

    Formal Guidance Competence
    Approximately 60 percent of the school-based teacher educators have guidance competence that grants them academic credits. A significant minority, around 40 percent, lack such competence. The percentage of those who reported not having this competence varies between institutions, ranging from about fifty to five percent.

    Experienced School-Based Teacher Educators
    Most of the school-based teacher educators have extensive experience in the education system. 78 percent have worked as teachers for over twelve years, while only two percent have worked as teachers for two years or less.

    Attractive but Time-Consuming to Be a School-Based Teacher Educator
    The majority of the respondents who conducted the survey find it attractive to be a school-based teacher educators. However, a significant minority feels they have insufficient time to attend to the students in school placement, and many of them express this in free-text responses.

    Collaboration with Universities
    Most of the respondents believe there is good collaboration with the university regarding the implementation of the supervised school placements. However, there are some deficiencies in the collaboration between the school-based teacher educators and the campus-based teacher educators. The majority report that they have little or no collaboration with campus-based teacher educators on research and on the development of education. Furthermore, more respondents feel that the collaboration with campus-based teacher educators is not good, than those who find it good. This is also reflected in the frequently used free-text fields, where many describe poor information flow or a lack of connection between what students learn at the university and what they are supposed to do in supervised school placements. In addition, there are challenges with concurrent and/or less practice-relevant work requirements during school placements. Many free-text responses express a desire for improved collaboration.

    Good School Placement Periods, but should be Longer
    The school-based teacher educators are satisfied with many aspects of supervised school placements, including the relevance, the experience it provides, the guidance they provide, and the students' achievement of learning objectives. However, most of them believe that the support from the campus-based teacher educators should be better. Many, although a minority, believe that the school placement periods should be longer. This is also evident in the free-text fields, where many write about the challenges of too short periods.

    Similarities in Responses
    When we categorize the responses based on the teaching subjects of the school-based teacher educators, there are no differences in response patterns. There are also few and minor differences when we categorize the data based on other background variables, such as school size, the size of the city/town where the school they work at is located, the grade they teach, their formal guidance competence, etc. In some cases, there are differences in response patterns based on which university the school-based teacher educators are affiliated with.

    Author: Pål Bakken

    Read the report (in Norwegian | pdf)

  • Evaluation of the initial teacher education for grades 1–7 and 5–10: Survey among initial teacher candidates

    Read the report (in Norwegian | pdf)


    Summary

    NOKUT is conducting an evaluation of the initial teacher education for grades 1–7 and 5–10 (GLU 1–7 and GLU 5–10). The evaluation is scheduled to be completed in the autumn of 2024. The report presents results from a survey among initial teacher candidates.

    The GLU Candidates
    The data includes those who completed initial teacher education between 2020 and 2022. This includes individuals who pursued GLU according to the new National Curriculum Regulations introduced after the transition to a 5-year program in 2017, as well as those who followed the 4-year GLU program under the old regulations. Thus, we can compare responses from these two groups.

    Little Difference in Response Patterns
    The report indicates that candidates from GLU 1-7 and GLU 5-10 respond similarly or nearly so to most of the questions. Additionally, the response patterns among candidates who graduated under the old curriculum are quite similar to those who graduated under the new curriculum.

    Candidates Who Don't Work as Teachers
    A minority of respondents did not work as teachers and were asked about the reasons for this. The most common reasons for not working as teachers in primary school are an experience/perception of a demanding work environment with limited resources, little support, and high work pressure. Other reasons given include not finding a job or not the desired job. Those who previously worked as primary school teachers are more likely to agree with statements about work pressure, limited resources, and little support compared to those without such job experience.

    GLU-teachers' Research Experience and School Experience
    Candidates believe that academic staff teaching at GLU educations have sufficient research competence, but they should have had more knowledge and experience from schools.

    School Placement
    Candidates are generally satisfied with their school placements and believe they gained valuable learning experiences during their school placements. Most are satisfied with the guidance they received from their school-based teacher educators during their studies, but fewer than half believe that their school-based teacher educators had a good understanding of the academic content of the teacher training program. Candidates feel they gained valuable experience in teaching their subjects, but the school placements did not provide them with sufficient experience in the broader responsibilities of the teaching profession beyond classroom instruction. A larger number of respondents disagreed than agreed with the statement that school placements are sufficiently long. In a widely used open-text question discussing the organization of school placement, many expressed a desire for more and longer school placement periods. Reasons included the opportunity to develop strong student-teacher relationships and practice collaboration, particularly between schools and homes.

    Academic Coherence
    On several statements about the academic coherence in GLU, more respondents expressed dissatisfaction than satisfaction. There are minor differences between GLU 1–7 and GLU 5–10 and between the 4-year and 5-year GLU programmes in these statements. An exception is the statement "the sequence and progression in teaching and school placement periods were well adapted to each other," where more candidates with the 4-year GLU program than with the 5-year GLU program felt that this was true.

    Course Offerings and Course Selection
    There are more candidates who believe that "specialization comes at the expense of breadth" than those who think the opposite. The majority believe that "there should have been more teaching subjects to choose from." A slim majority agrees to a great extent that "there are too few subjects to choose from," but a significant minority disagrees. Furthermore, most candidates believe there is "too much specialization in the master's subject," and that "the content of the study subjects is too narrow compared to the needs in schools." Candidates are divided in their views on the statement "my subject circle is/becomes too narrow"; 33 percent disagree to a small extent, 37 percent to some extent, and 30 percent to a large extent. Candidates from the 5-year GLU program agree with the statement more than those from the 4-year GLU program. In a subsequent open-text field, many candidates mentioned the desire to choose more subjects, and many highlighted a mismatch between the number of subjects they could choose in their studies and the need to teach multiple subjects in primary school.

    Outcome of the Master's Thesis Work
    Those who wrote a master's thesis as part of the 5-year GLU program were asked to evaluate the extent to which their master's thesis work benefited them in various areas. Respondents had mixed opinions on this, with some stating they gained significant benefits while others reported less benefit. Half of the respondents strongly agreed that their master's thesis work helped them "realize the importance of being academically and pedagogically updated," while a small minority strongly disagreed. Many also believe it gave them "academic confidence." Some were less likely to agree that it "provided me with valuable insights and knowledge that prepared me well for the role of a teacher." The statement about "reading research and extracting central information" received strong support, with the vast majority in agreement. In a subsequent open-text question, a majority stated they gained relevant learning outcomes. Many mentioned gaining research competence and information processing skills. There were also some who gained research competence but unequivocally stated that this competence was not relevant to their current job as teachers. Furthermore, quite a few candidates mentioned gaining increased depth in their master's subject, while a significant minority reported not gaining relevant learning outcomes from the master's thesis at all.

    Utilization of Skills from Master's Thesis Work
    Most candidates indicated that they use the skills gained from their master's thesis work to a limited extent in their role as teachers. For example, only a quarter reported using them to a great extent in teaching, planning, and discussions.

    Preparation for the Teaching Profession
    Regarding a statement about candidates being "well-prepared for their first job as a teacher," a quarter disagreed to a small extent, a quarter largely agreed, and half agreed to some extent. Candidates are more positive about whether they "have the necessary knowledge, skills, and competence to become a good teacher" and "have a solid foundation for the further development of their practice and the teaching profession."

    Learning Outcomes
    Respondents were asked several questions about their learning outcomes. Many candidates feel they have gained sufficient learning outcomes in teaching competence and what can be described as research competence. However, most feel they have not gained sufficient learning outcomes in "the transition between different grades" and "initiating measures for students in difficult life situations."

    Authors: Pål Bakken og Erlend Langørgen

    Read the report (in Norwegian | pdf)

2022

  • Integrated Secondary Teacher Education

    Read the report (in Norwegian)


    Abstract

    This is the final report from NOKUT’s evaluation of the 5-year master’s programme Integrated Secondary Teacher Education. The evaluation has run from 2020–2022 and is a part of NOKUT’s work to contribute towards quality assurance and enhancement in education. The aim for this evaluation has been to generate knowledge about quality in Integrated Secondary Teacher Education and contribute to enhancement.

    The evaluation approach
    NOKUT has emphasised that the evaluation should be useful to people who work with Integrated Secondary Teacher Education, and the project has therefore facilitated involvement, participation and the exchange of knowledge and experience during the evaluation. This includes dialogue about the makeup of the expert panel, the selection of evaluation themes and the formulation of evaluation questions.

    This is the evaluation’s expert panel:

    • Inga Staal Jenset, associate professor at the University of Oslo. Subject area: Pedagogy and teacher education. Jenset leads the expert group.
    • John Brumo, professor at NTNU. Subject area: Scandinavian literature.
    • Erling Hans Eiterjord, Head of School at Alstad lower secondary school in Bodø.
    • Ingunn Folgerø, Head of School at Bryne upper secondary school and leader of the Norwegian teachers’ council for ethics.
    • Đula Ganić, student representative and now graduate from the Integrated Secondary Teacher Education programme at the University of Agder.
    • Mogens Allan Niss, professor emeritus at Roskilde University. Subject area: Mathematics and mathematics education.
    • Tine Sophie Prøitz, professor at the University of South-Eastern Norway. Subject area: Education policy.

    The expert panel limited the scope of the evaluation to five evaluation themes that are relevant to quality enhancement in Integrated Secondary Teacher Education. These five themes were selected based on input from NOKUT’s dialogue with HEI staff and students as well as other stakeholders, previous evaluations and other reports, relevant policy documents and academic scholarship. The themes are:

    1. National governance and institutional autonomy
    2. Leadership, organisation, and collaboration in Integrated Secondary Teacher Education
    3. Coherence in Integrated Secondary Teacher Education
    4. Integrated Secondary Teacher Education students’ professional identities and belonging
    5. Novice secondary teachers’ competency and further development  

    The evaluation has examined these five themes at each of the eleven institutions that offer Integrated Secondary Teacher Education.

    The evaluation data includes the HEI’s self-assessments and reflections, digital site visits, data from the National Database for Higher Education Statistics (DBH) and surveys among former and current students, recent graduates, school leaders and HEI staff who teach Integrated Secondary Teacher Education students.

    In this final report, the expert panel will discuss different approaches to Integrated Secondary Teacher Education, identify different strengths and challenges, and facilitate the sharing of knowledge and experiences across the participating institutions.

    About Integrated Secondary Teacher Education
    Integrated Secondary Teacher Education programmes qualify graduates to work as a teacher in lower and upper secondary school. This education has seen increasing student numbers, from admissions of nearly 400 new students in 2003 to 1380 students in 2019. In the autumn of 2021, there were over 5000 registered Integrated Secondary Teacher Education students in total in Norway. In addition to increasing admissions figures we also see increasing completion figures. The study programme is offered at nine universities and two university colleges.

    In 2013 the Ministry of Education and Research introduced a new national curriculum framework (rammeplan) that encompassed all five years of the Integrated Secondary Teacher Education. The expert group on the teacher’s role (Ekspertgruppa om lærerrollen) notes that this marked the first use of framework plans at the master’s level in Norwegian education and the purpose of the plans were to facilitate quality enhancement in teacher education for years 8–13, adjusted to schools’ needs for teaching competency. The study programmes were to become integrated, research-based and professionally oriented, with stronger interplay between subjects, subject didactics, pedagogy and practical training. The professional subject (profesjonsfaget, which includes pedagogy, subject didactics and practical training) was given a particular responsibility for maintaining maintaining connections between these different components and for giving students a shared identity as schoolteacher. The study programme should now include 100 days of supervised and assessed teaching practice, divided over at least four years of study, with progression. Thus, the five-year study programme was to be far more integrated, include more extensive practical training and facilitate student development of professional identities as secondary school teachers. Additionally, modelled on Primary and Lower Secondary Teacher Education, national guidelines for Integrated Secondary Teacher Education were also developed to ensure a commitment to shared quality standards for strong teacher education (NRLU, 2017, p.2).

    The distinction between the professionally oriented and the academic, and the turn towards a more professionally oriented focus also for the Integrated Secondary Teacher Education, forms an important backdrop for this evaluation.

    Evaluation results and recommendations
    The final report provides a two-part presentation of evaluation results:

    • In chapter 3 the panel presents evaluation results at a national level, with recommendations for national authorities, HE institutions, and schools/school owners for further work towards quality enhancement of Norwegian Integrated Secondary Teacher Education.
    • Chapters 4–15 describe results and recommendations connected to evaluation themes 2–5 for each of the eleven HEIs offering Integrated Secondary Teacher Education.

    The evaluation demonstrates that most of the Integrated Secondary Teacher Education programmes in Norway are doing systematic and good work to achieve educational quality.

    The evaluation also shows that the programmes share many of the same challenges, and that these are in part well-known and recurring challenges in Norwegian and in international teacher education. We also see that some of these challenges are caused by or strengthened by the characteristics of Integrated Secondary Teacher Education, as described in the introductory chapter to this report. All teacher education programmes are complex but, for several reasons, Integrated Secondary Teacher Education programmes have traditionally been particularly complex. In addition to being organised across the HEI and their school partners, it is usually also organised across different academic disciplines, institutes, and faculties. This strengthens the quality of education in the subject areas, but also creates challenges with coordination and collaboration between subject areas, the professional subject, and practical training. The national framework plan’s requirement to distribute practical training across four of the five years of study is also difficult to balance with such a structure. The study programmes are crammed with coursework modules, practical training and additional optional learning activities, and at times the institutions struggle to provide adjustments in such a way that the many components in the study programme can form a unified and connected experience for the students. The panel has therefore found it important to balance the relationship between what might be ideal solutions for the study experience and learning of the students with existing national and local premises and conditions for Integrated Secondary Teacher Education.

    Chapter 3 points to four areas that are particularly important for further quality enhancement work in Norwegian Integrated Secondary Teacher Education. Below we summarise the national recommendations connected to each of these four areas. Recommendations for individual institutions can be read in chapters 4–15.

    Improved balance between national governance and institutional autonomy

    In sum, the panel recommends that national authorities:

    • reduce national governance of Integrated Secondary Teacher Education, particularly in connection with the requirement to distribute practical training across four of the five years of study, and the mandatory bachelor’s dissertation in year three. This should be considered within the context of the Ministry’s work on revising the national framework plans for a less extensive governance of teacher education. The panel majority believes that the national authorities can stress the importance of regular practical training without quantifying this in detail (The panel minority, member Prøitz, does not support the recommendation to ease the requirement to distribute the practicum across four of the five years of study. Prøitz believes, in line with recognised research on practical training, that regular practical training throughout the study programme is of great significance for the professional qualification, and that it is unfortunate when too much time passes between each practical training period, in line with what students, school leaders and teachers, as well as HE staff members at some of the study programmes noted during site visits.)
    • complete an impact assessment of separate credits for the practical training component in teacher education in general
    • complete a national assessment of the financing of Integrated Secondary Teacher Education
    • trial alternative methods for documenting applicants’ competencies for admission to Integrated Secondary Teacher Education, in light of the forthcoming Norwegian Official Report (NOU) on HE admissions regulations
    • strengthen the work to recruit teachers to subject areas where there is a lack of qualified teachers

    The panel recommends that the HEIs:

    • review which subject combinations they offer, considering needs in schools and in society, and the allocation of study places in the different subject areas
    • prioritise local work on course curriculum (studieplaner) to better identify and take advantage of existing flexibility in national policy documents, and in turn the institution’s particular strengths
    • increase the degree of collaboration with other institutions to learn from each other’s experiences before introducing changes to study programmes, for example through the establishment of networks for institutions that offer Integrated Secondary Teacher Education
    • make use of existing knowledge before introducing new quality enhancement initiatives

    Establishing a culture for Integrated Secondary Teacher Education and greater focus on the role of the programme director (studieprogramleder)

    In sum, the panel recommends that national authorities:

    • define and make explicit the obligation of school owners to contribute to the students’ practical training
    • consider new solutions for financing the practical training, in light of the National Forum for Teacher Education and Professional Development’s (NFLP) work on the practical training and the 2025 teacher education strategy

    The panel recommends that the HEIs:

    • clarify and anchor the existence of and justification for Integrated Secondary Teacher Education at the highest levels of leadership within the institutions, and highlight the strategic position and priority of these study programmes at the institutions
    • facilitate the development of a culture for Integrated Secondary Teacher Education, where relational work across actors, including schools, is important, and where there are opportunities for dialogue about the key terms, content, and organisation of Integrated Secondary Teacher Education
    • focus on the role of the programme director (studieprogramleder) and ensure that the programme director has the resources and authority required to fulfil their responsibilities. This involves institutional changes that give the programme director increased authority
    • establish relationships with school partners that ensure collaboration and interplay in the development of the programme content, including the practical training

    Degree of coherence and student workload

    In sum, the panel recommends that national authorities:

    • take the programmes’ complexity and demand for coordination and collaboration into consideration when reviewing the financing of Integrated Secondary Teacher Education
    • differentiate expectations of coherence in Integrated Secondary Teacher Education, in comparison with most of the Primary and lower secondary teacher education programmes and the Postgraduate certificate in education (PPU)
    • discontinue the use of the term "integrated" in the national curriculum framework for Integrated Secondary Teacher Education

    The panel recommends that the HEIs:

    • review the local allocation of resources to implement necessary initiatives to create coherence
    • continue the work towards coherent study programmes, and the work of clarifying what students can expect in a complex five-year master’s programme
    • make use of school partners’ competency in work on the study programme
    • differentiate learning activities and assessments in shared modules to better meet the respective needs of Integrated Secondary Teacher Education students and other students
    • review the students’ full course of study. This means avoiding add-on initiatives when possible and instead incorporating such learning activities into the ordinary course of study. This can be seen in connection with the previous recommendation.

    The relevance of subjects in Integrated Secondary Teacher Education

    In sum, the panel recommends that national authorities:

    • carry out mapping and quality assurance of the Integrated Secondary Teacher Education students’ practical training
    • assure the quality of the system for professional mentoring of novice teachers, and ensure sufficient resources for achieving the required number of qualified mentors in schools

    The panel recommends that the HEIs:

    • work on adapting relevant subject modules (disiplinemner) to Integrated Secondary Teacher Education students, perhaps in collaboration with colleagues from the professional subjects and school partners
    • work on the relevance of the professional subjects in collaboration with school partners
    • enable Integrated Secondary Teacher Education students to write their master’s dissertations in subject didactics, and establish a system in which schoolteachers co-supervise master’s dissertations
    • improve the quality assurance of the Integrated Secondary Teacher Education students’ practical training
    • establish specific courses in "teacher education pedagogies" for teacher educators who require basic HE pedagogical competency

    The panel recommends that schools:

    • improve quality assurance of the mentoring of student teachers and novice teachers
    • prioritise ensuring that mentors gain professional qualifications in mentoring

    Authors: Chapter 1 and Chapters 3–14 have been written by the expert committee. Chapter 2 was written by NOKUT.

    Read the report (in Norwegian)

     

2018

  • Joint evaluations of research and education – a pilot study

    Read the report (pdf)


    Conclusions

    NOKUT and the RCN have received the reports from the three disciplinary panels, and structured them as sub-chapters in this overall interplay report. In this concluding chapter, we highlight some common patterns, point to differences between the disciplines, and discuss the implication of the panels’ findings. Last, but not least, we will present our own reflections about the evaluation and the road ahead for future evaluations of research and education quality in Norwegian higher education. It should be noted that the observations and corresponding implications raised in this chapter are made by NOKUT and the RCN, and they should not be attributed to the panel members responsible for the previous chapters 3–5.

    Before presenting the national-level findings, two reservations need to be pointed out. Firstly, it is important to note that the interplay evaluation in practice became an assessment of the influence that research quality and research activities has on educational quality and not the other way around. There are many reasons for this, and the perhaps most important one is that this line of influence – going from research to education – is intuitively assumed by most stakeholders. The literature review produced by NIFU (see chapter 2) identified several interplay quality dimensions and these where all centred around how research quality impact educational quality in higher education and not vice versa. As a result, the identified interplay indicators did not facilitate assessments of how education can be said to influence research.

    Secondly, it is important to highlight that this is an evaluation of three traditional university disciplines, all with long-standing research traditions. The three disciplines are to a large extent theory-driven, and they are to a lesser degree influenced by the rapid evolution of professional practice when compared to more practice-oriented disciplines such as engineering or nursing. It is therefore necessary to underline that the generalizability of the conclusions highlighted in this chapter are limited, and also that the extent to which one can use the same evaluation model and quality dimensions for other disciplines in higher education is uncertain. This is particularly important to keep in mind when considering less theory-driven and more practice-oriented disciplines. Dimensions such as the interplay between practice and education, the importance of practice-oriented teaching and the value of having lecturers from the field of practice have hardly been considered in this evaluation, although they are arguably highly important dimensions for the quality of education in many practiceoriented disciplines.

    Overall assessment of interplay across political science, sociology and economics
    An important finding deriving from the first step of the interplay evaluation is that there is a large degree of overlap in the assessments of research and education. That is, institutions that receive a high score in the research evaluation also receive favourable education evaluations. With few exceptions, this pattern repeats itself across disciplines and institutions, indicating a positive relationship between research quality and educational quality. The different chapters suggest, in various ways, that a causal relationship might exist between the two activities, i.e. that high-quality research leads to high-quality education, but the actual mechanisms at play remain uncertain.

    The panels are also joint in highlighting significant differences between the BA and the MA degrees concerning research orientation in general, and in the emphasis put on research methods and size of theses in particular. The political science and sociology panels considers this gap too large, and argue that the BA level should include more methods training and that the BA theses should occupy a larger space in the programme. This is so, as argued by the sociology panel, because these are the educational activities in which research skills are developed and trained most evidently. In addition to providing sufficient amounts of methods training, the panels encourage the institutions to further integrate methods and research skills with more theoretical and/or topical courses. In other words: the BA should be more research oriented, the development of research training should start early in the study programme, and the institutions should attempt to integrate methods courses with other courses in the programme.

    Another interesting observation is related to faculty size and the scale and scope of the study programmes offered. The panels observe that smaller faculties have problems offering educations with sufficient breadth of courses and areas of specialisation while at the same time ensuring that the study programmes are research based. Institutions are then faced with a choice between specializing the degree towards one or more sub-fields, ensuring a good match between the faculty’s research areas and the educational offer, or keeping the broad disciplinary study program notwithstanding that some parts of the program may not be based in a research environment at the institution. The panels have recommended a larger degree of collaboration between institutions and/or specialization at the individual institutions in order to overcoming this challenge.

    Implications

    The positive relationship between research quality and education quality
    The empirical data generated from the research, education, and interplay evaluations indicate that a positive relationship exist between research quality and education quality in the three disciplines included in the evaluation. The format of the evaluation – including available data, time and resources available – does not allow for making strong causal claims, and the panels themselves do not speculate on the causal mechanisms that link research quality and educational quality. This was also not the aim of the evaluation, which had the more modest ambition of knowing more about the synergies between the two activities. However, looking at the three disciplinary assessments of interplay, we find it both relevant and important to start a discussion about possible explanations.

    The first, and perhaps most intuitive explanation that stand out from the report is that high quality research environments ensure research based education. That is, institutions with high-quality research environments are more likely to have the knowledge and resources necessary to ensure research-based study programmes in the broad understanding of the word. This means that the institution can deliver education that is both research-tutored, research-based, research-led, and research-oriented (Healey 2005, see page 10, this report). Put differently, the education contains elements of all, or close to all, of Kyvik and Vågans’ seven interpretations of research-based education (see page 10, this report). In contrast, education offered at institutions that score lower on research quality is likely to be researchbased in a narrower sense. This explanation implies that the inherent characteristics of a high quality research environments ensure research-based education, which in turn leads to high-quality education.

    There are, however, explanations that do not centre on the high quality research environment per se, but rather on associated characteristics. We know from the data material for the research and education evaluations that the institutions with high-quality research are the institutions that attract the strongest students measured in grade points from higher secondary education. These students may raise the level of the education through more engaged participation, demands and results. As such, it is not the high quality research that causes high quality education, but the highly selected students. It is also notable that, although with some minor exceptions, the institutions that have high research quality have longer traditions in offering educations in the three disciplines political science, sociology and economics. It is not unlikely that experience over time has a positive influence on the quality of the study programme.

    These suggested explanations are not mutually exclusive, and it is likely that they reinforce each other. The combination of a high-quality research environment, an experienced and resourceful educational leadership and study administration, and students with the necessary motivation and skills to both advance their own knowledge and to contribute to the development of the study programme itself through critical engagement is likely to significantly strengthen the quality of the education.

    The importance of researchers as lecturers
    In order to ensure a broad research base for study programmes, it appears critical that involved lecturers are active researchers themselves. This is not to say that lecturers must be researchers to ensure research-based education, as there are many roads to ensure a (narrow) research base. However, if a programme aims at delivering high-quality education, which is likely to necessitate a research base in the broad understanding of the term, ensuring that the lecturers are active researches themselves seems decisive.

    To highlight the seeming decisiveness of researchers as lecturers is not to say that all researchers are automatically good lecturers. Being a good lecturer is likely to also depend upon other factors, such as pedagogical training, communication skills, sufficient time for preparation, adequate educational facilities and a supportive network of peers/colleagues. However, having research experience – and ideally being an active researcher – appears to be a necessary condition (although not sufficient in and of itself) for providing research-based education in a competent way, according to many observations highlighted in this report.

    Faculty size
    According to the panels, sociology, political science and economics education require a sizeable and research active faculty in order to provide the breadth of substantive courses, methods training and specialization that these disciplines require. This does not necessarily mean that smaller faculty size should prevent an institution from offering the degree, but the organization and structure of the degree should mirror the faculty’s research competence. Not taking research competence into consideration when planning the study programme risks ‘stretching thin’, i.e. study programmes with uneven quality of courses.

    As already mentioned, the panels raised several options for amending small faculty size/limited research base, such as cooperation between institutions or focusing the degree towards one or more sub-disciplines.

    Boosting research orientation at the BA level
    Another important assessment the panels have made, is that there is a real need to boost the research orientation of the bachelor programmes that have participated in this evaluations. The panels suggests three interlinked ways to do this. First, the programmes need to include more methodology training early in the bachelor programmes. The programmes should also integrate more methodology training into the more substantive courses, and finally, many of the programmes should increase the demands for the bachelor thesis. An important reason for this suggestion is that methodological training is more than an academic exercise, it is a critical way to increase the students’ generic skills, which make candidates more attractive on the labour market.

    To NOKUT this is particularly interesting when one considers the regulations institutions must meet to accredit bachelor, master and PhD programmes. Though by law, all higher education must be research based, the requirements regarding the research competence of the faculty providing a bachelor programme are lower than for a master degree. The assessments made in this evaluation for the disciplines of political science, sociology and economics clearly question this logic.

    Lessons learned for future interplay evaluations
    NOKUT and the RCN can draw several lessons based on this pilot evaluation.

    First, it is clear that to assess the quality of education one needs to assess the broad research base of said study programme. It is necessary, but not enough, to look at the individual lecturers’ education or CV. The evaluation also needs to assess the way in which the programme is designed, the curriculum, the amount, content and timing of methodological training, and the nature of the bachelor and master theses.

    Second, in order to evaluate more efficiently research and educational quality and the interplay between the two, the evaluation should be more integrated. In the current evaluation, the units that have been evaluated in the research evaluation and education evaluation are not the same. That is, in the research evaluation, the RCN have evaluated the different disciplines at the institutional level, while NOKUT has focused on the different study programmes. In the future, the units should be better aligned. One should also consider using one expert panel for the entirety of the evaluation. As is described above, the interplay evaluation has been done by one member of the education panel and one member of the research panel. The panellist themselves have indicated that it would be beneficial for one panel to assess research, education, and interplay quality all together, rather than to split it up into three different evaluations with different panels.

    Third, NOKUT and the RCN need to think about how we in the future can assess the impact education has on research and not only the other way around.

    Finally, if the model piloted in this project would be set in motion in a new large-scale evaluation, NOKUT and the RCN, for the benefit of the participating institutions and panel members, need to improve the coordination of the evaluation.

    Read the report (pdf)

  • Educational quality in economics in Norway

    Read the report (pdf)


    Conclusion: overall assessment

    Our overall assessment of economics in Norway is positive. The programmes are of good quality, but still somewhat traditional and inward looking. In this concluding chapter, we aim at formulating a series of rather generally applicable conclusions (we refer to the previous chapters for the individual institutes and programmes). This of course implies that some of these conclusions hold more strongly for some than for other programmes. The conclusions form the basis for the recommendations the panel provides in chapter 9.

    A first general conclusion is that most of the programmes, especially at the master level, are relatively small. This holds especially for the universities in the more peripheral regions of Norway. The degree of specialisation is rather limited, although the smaller universities typically seem to specialise more and have stronger connections to the local community. Some of the programmes are so small that the panel wonders whether the programmes can offer the quality and diversity that students desire and need for developing academically. 

    A second observation is that the self-assessments did not reveal much about strategic plans. Clear visions of how the schools want to develop and why are missing. Similarly, clear visions of new developments in the field how to integrate new developments in the educational programmes are limited. The limited strategic thinking occurs in a period of comprehensive discussions about the quality of education in economics in view of the Great Recession. Furthermore, the current setting is influenced by the perceived need to pay more attention to behavioural economics in our programmes, for example the CORE project; (see, e.g., The Economist, September 23, 2017). This corresponds with the earlier observation of traditional programmes offered in Norway.

    A third conclusion is that Norway apparently has not (yet) fully adopted the principles that were agreed upon in Bologna. The panel considers the five-year programmes as peculiar leftovers from the former system and has the impression that differences in terms of quality between the 5 and 2+3 year programmes are rather subtle. Content-wise, most programmes have an almost exclusive focus on economics. Based on the useful insights to be gained by students from business economics and business administration, this is a limitation. Moreover, this is not in line with the Bologna principles, which state that bachelor programmes should be relatively broad in nature.

    A fourth conclusion is that Norway seems to lag behind regarding its internationalisation ambitions when compared to many other countries in Europe, which surprises the panel given the small and open Norwegian economy. The limited focus on internationalization is evident in the mainly Norwegian language courses at the universities, and in the limited focus on international accreditations, which foreign students often consider to be an important indicator of high quality.

    A fifth observation is that we have seen limited evidence of distinct educational leadership. The panel has not identified organisational structures where programme directors with a strong academic reputation, and the ambition and task to continuously develop and improve his/her programmes. The panel has the impression that this may well be one of the important reasons for the traditional orientation of the programmes. In the panel’s opinion, there is limited evidence of coherent programme structures, which unfortunately leads many programmes to resemble a collection of high quality courses rather than integrated programmes tuned towards a higher level of achievement. We emphasise that this impression is based on the material we received, and it is one of the conclusions that we had liked to substantiate based on discussions with representatives from the schools. However, this was not possible in the context of this pilot. We also saw limited evidence of explicit recognition of the most innovative and best lecturers. The panel detected little evidence of career paths for top lecturers, even though educational quality is a criterion in the evaluation of scholars. There seems to be a bias towards recruiting strong researchers who then are automatically expected to be excellent and active in teaching.

    A final observation is that the integration of research and education is mainly established in the theses projects at the end of the programmes. The panel is of the opinion that a further and earlier integration of research experiences in the programmes is desirable.

    Read the report (pdf)

  • Educational quality in political science in Norway

    Read the report (pdf)


    Conclusion – Overall assessment of the disciplinary area 

    General comments
    The evaluation of political science education in Norway encompasses programmes at seven different universities. While all of the included programmes are broadly conceived within the discipline of political science, there is wide variation in the titles and content of the programmes. Overall, a main distinction can be made between programmes which use the political science label and include introductions to all or most major sub-disciplines within the field of political science and programmes which have a more narrow focus, mainly related to public administration (and in the case of the University of Bergen also a programme specialising on comparative politics). In addition, the programme at the Oslo and Akershus University College of Applied Sciences to a large degree focuses on the practical use of public management skills, while the other programmes are more research oriented.

    Correspondence between the program title, the learning objectives and the course content is provided in most of the programmes. It is therefore not necessarily problematic that the programme titles vary. It is, however, worthwhile to consider whether coordination between the institutions could benefit potential students who may find it difficult to distinguish between MA’s in ‘administration and organisation theory’, ‘political science and management’ and ‘public management’. Apart from the main distinction between general political science programmes and programmes focusing on public administration, the panel has the impression that the diversity in the labelling of the programmes reflects path-dependence rather than major substantial differences programmes.

    In addition, it is not clear to the panel to what extent the programmes are targeting the same or different labour markets. From the examples available at the webpages of the universities, it appears that the typical jobs, which candidates from different programmes obtain, are largely similar. However, a more systematic analysis of the patterns of employment would be useful regarding the institutions’ development of their programmes and for prospective students.

    The strategic situation of the programmes
    The included programmes vary in size with some programmes accept about 40 new BA students each year and others more than 150 new students. There is a clear geographical pattern in the number of first choice applicants, where the universities located in the Oslo, Bergen and Trondheim area are able to attract a much larger number of applicants than those located in Agder, Stavanger and Tromsø. It is particularly noticeable that the University of Tromsø has fewer applicants than the number of study places available. Among the other universities, the University of Bergen has the lowest number of applicants per study place, which is largely a result of the high number of study places offered in the programmes at Bergen.

    This situation reflects how the universities are in very different strategic situations. For the universities located in the larger cities it is not only easier to attract students, but also easier to attract highly qualified international staff members. These universities are therefore able to provide very good programmes which are on par with good international political science programmes in terms of the faculty teaching and the curriculum. The strategic challenge is to maintain the quality of education by paying continuous attention to faculty recruitment, programme development and advancement in teaching and assessment methods.

    For universities with a less favourable geographical location, a main strategic challenge is the recruitment of qualified staff and students. The panel recognizes that several of the institutions are aware of these challenges and have taken crucial steps to meet them. Among these are specialisation in areas where it is possible to build up a critical mass of staff and cooperation with other disciplines. The panel recommends paying even more systematic attention to this challenge and discussion among institutions in similar strategic situations.

    A common challenge for the programmes seems to be relatively high dropout rates (above 40 percent). Also, at most universities the students take only 40 credits per year, which is substantially lower than the 60 credits constituting a full year of work.

    The design of BA and MA programmes
    The structure of the BA and MA programmes are relatively similar across the different universities. The typical BA programme consists of a number of mandatory courses within the field of the programme and a number of elective courses. The BA programmes typically include some methods training, but from an international perspective there is relatively little focus on research methods at this level. It is also noticeable that the number of ECTS allocated to the BA thesis is only 10 at some programmes.

    In contrast, the MA programmes are generally very research oriented. Most programmes have several mandatory courses on research design and methods, and the MA thesis is a very prominent part of the MA programmes with up to an entire year assigned to activities related to the MA thesis. The number of substantive courses at the MA level is typically rather low, which reflects the emphasis on design and methods. From an international perspective, the contrast between a BA with little emphasis on research design and methods and a MA programme where research design and methods represent the central focus, must be seen as a crucial point of attention. First, students obtaining a BA degree will have received limited methods training. This limits their ability to independently assess the academic literature and to finish with a BA thesis by utilizing research methods at a sophisticated level.

    Second, the research intensive MA programmes may prepare students better for pursuing a research career than for the labour market in the private and public sector outside of academia. While the panel believes that the skills obtained through methods training and in regard to writing a MA thesis are also transferable to the labour market, a MA with a more balanced focus on research competencies and substantive insights appears more targeted on the general labour market. Third, the BA structure may serve as a barrier to international mobility. For example, a minimum of 15-20 ECTS of methods training is required in order to obtain admission to a MA programme in political science at Danish universities.

    Teaching and assessment methods
    With respect to teaching methods, the impression of the panel is that the teaching and assessment methods used in political science programmes are fairly traditional. Here, it is particularly crucial to note that the material available for the evaluation does not allow for a very thorough assessment. Nevertheless, according to the Studiebarometer and the information provided in self-assessments the emphasis of most programmes is apparently on lectures and seminars. There is little evidence of the use of new pedagogical methods including the use of digital instruments. Also, in many programmes traditional school exams are used in a high proportion of courses. Overall, there seems to be a potential for innovation in teaching and assessment including the use of digital learning instruments.

    Read the report (pdf)

  • Educational quality in sociology in Norway

    Read the report (pdf)


    Conclusion – Overall assessment of the disciplinary area

    General remarks, initial competence
    All BA-level sociology programmes across the country require the Higher Education Entrance Qualification. For the MA programmes, there is some slight variation in the requirements, with some departments demanding a BA degree with grade B and some specifying a methods requirement. In general, students who choose sociology programmes have moderate grade point requirements, and some programmes have few applicants. It can be assumed that the initial competence of BA students in some programmes is satisfactory, whereas some programmes really struggle. Almost all programmes offer a range of activities both for promoting the programme and for supporting incoming students, and several programmes collaborate with student organisations. Most of these activities target BA students. Some MA programmes offer various support also for MA students, but in general, the self-assessments lead the panel to believe that the frequency and intensity of student support decline as students progress from one level to another. Presumably, though, PhD students in Norway will be able to take advantage of the same structures as faculty members.

    General remarks, programme design
    The programme design appears highly similar across institutions. The mainstream BA programme starts with an Introduction to Sociology course (20 credits) together with Ex. Phil. and Ex. Fac., followed by topic-oriented courses (e.g., family, migration), one or two research-methods courses, a substantial number of specialisation/elective courses and a BA thesis at the end. In the MA, this logic is usually extended. The ability to think critically and independently is high on the desired learning outcome, and almost all programmes succeed in realising this goal – students unanimously indicate this as a strong point of their education.

    The impression of the panel is that, in general, students in Norway become very familiar with a variety of sociological topics, qualitative and quantitative methods, and sociological classics, and have ample room to design their own study programme – even up to following subjects outside sociology. According to the panel, however, this mainstream programme structure also has several serious omissions. First, BA and MA programmes are not very cumulative. Programmes very often consist of stand-alone courses, in which students can participate without prior knowledge – this happens even in third year BA courses. It therefore seems that the aim to provide a very diverse range of methods and topics comes at the expense of providing a more coherent, in-depth, advanced and cumulative programme. Second, and related to the first point, the panel believes that programmes pay insufficient attention to research methods. Most programmes are thin on research methods courses, in particular quantitative methods and statistical techniques. The panel holds the opinion that methods skills need to be more integrated into the substantive courses rather than something to be learned in the stand-alone methods courses. Students would benefit from methods and statistics courses earlier on, preferably in the first semester, as this creates opportunities to integrate and apply these methods in the follow-up (more substantive, topic-related) courses (e.g., inequality, migration, etc.). In order to acquire the methodological and statistical tools to do sociological research, students need not only learn about these methods and techniques in a stand-alone methods course, but also to apply these in the substantive courses (which now typically focus on theory and argumentation). Third, it also strikes the panel that most programmes disproportionally pay attention to classical sociology at the expense of modern sociology. Classic work takes a central place in many courses throughout the programme (e.g. ‘introduction to sociology’, ‘sociological classics’, and the many topic-oriented courses). Even courses on ‘modern sociology’ or ‘contemporary social theory’ typically cover sociological research from the 1960s, 1970s and 1980s (e.g., Foucault, Bourdieu) and omit recent sociology. The panel identifies ‘two sociologies’: on the one hand, the theories, methods and findings addressed in contemporary international sociological research, and on the other hand, the sociological studies discussed in education. The panel believes that in Norwegian sociology programmes, the state-of-the art of sociological theories, methods and findings need a much more prominent place.

    General remarks, teaching and assessments methods
    Most of the teaching and learning activities and assessment methods seem to be rather traditional, but with some notable exceptions. There is a clear national picture in which lectures, seminars and written assignments predominate. This picture can often hide some of the variety and innovation that the panel found happening within individual courses and programmes. Nevertheless, increasing the variety of teaching and assessment methods, and encouraging innovation should be a national priority for sociology programmes. In this respect, institutional incentives and resources are key. The panel found a strong connection between variety and innovation and the provision of institutional incentives (funds, award scheme, merit systems). Teaching status and recognition is also important for teaching innovation, and this needs to be fostered in the culture of the institution. The panel found that research and teaching are still often pitted against each other. Developing a focus upon the value of researchled teaching would be useful.

    What stands out nationally when it comes to teaching and assessment methods in sociology, is the relative absence of e-learning or blended learning initiatives. It could be either that this is an area with plenty of room for improvement and expansion, or that it is so ingrained in what the universities do, that it is not even considered worth mentioning. The panel assumes, however, that considering low student awareness of the use of digital work methods, the former is more likely. An increased focus on the didactic gains in using educational technology could be a general focus for future development initiatives. Since this is a pilot project, future assessment exercises may also choose to ask more specifically about the use of educational technology. The use of e-learning technology would be particularly appropriate for the institutions facing geographical and structural challenges.

    General remarks, learning environment
    Studiebarometeret provides some insight into the way that students perceive their education. However, the panel is well aware of the low response rate, the lack of a variance measure, and the lack of a nonresponse analysis. Consequently, we suggest that these numbers be interpreted with great caution. In addition, the panel is under the belief that a student satisfaction survey is a sub-optimal indicator of the quality of the learning environment.

    Even though students are generally content with library services and ICT, it should be noted that the Studiebarometeret results indicate that in other respects the learning environment is an area for improvement nationally. Beyond the appropriate study facilities and ICT support, a good learning environment should provide both social and academic support. This requires encouragement of student activities that promote student community support and engagement in activities that promote interactions between staff and students. It also requires integration of research and teaching, where state-of-the-art research is integrated in education and an environment where students participate in various academic meetings, conferences, seminars, etc.

    General remarks, educational competence
    Educational competence of staff is generally a well-conceived aspect of the quality of the sociology programmes in Norway. Most institutions incorporate teaching credentials as an important criterion for hiring and promoting staff. Proven teaching competence and trial lectures are often part of the selection procedure, although teaching abilities are sometimes seen as less important than research output. Additionally, most institutions organise activities to further increase teaching competence. The panel recommends that staff make more use of such teaching facilities, as this is not common practice in all institutions. Funds need to be available to develop teaching competence. As good practices to further develop teaching competence the panel noted peer-to-peer feedback (staff evaluating each other), evaluations of courses, regular teaching seminars, and rewarding good teaching.

    General remarks, achieved learning
    The assessment of achieved learning is based primarily on the student survey, Studiebarometeret. Since this survey has low to moderate response rate, and an unknown item-specific response rate, the information is questionable – even more so for small programmes where the number of respondents make the results statistically meaningless. In all cases where the number of respondents to the survey is below 10, we have refrained from using the information. The weak reliability of Studiebarometeret is unfortunate since such surveys are instrumental for benchmarking and quality assurance of higher education, being useful both for programme development purposes and for student influence.

    For sociology MA students in Norway, the scores in Studiebarometeret are largely satisfactory, while there is room for improvement for BA students. Sociology students in Norway are reasonably content with the skills they receive in theoretical knowledge, methods, critical thinking, communication and independence. However, a general issue of concern according to the panel is that sociology students in Norway feel that their education does not provide discipline specific skills. This is highly unfortunate. The panel is of the strong belief that sociologists in general should receive training in methods, analysis, and critical thinking that is in high demand. It is an important task for Norwegian sociology to reframe the self-image of sociology as pre-occupied with academic discourse into the image of a discipline that has the means to address and understand challenges in contemporary society across many dimensions. In Studiebarometeret, students also express concern about their experience with ‘research and development work’. Possibly, this reflects the need to pay more attention to research methods and analysis throughout the programme (and not only in a stand-alone method course).

    Studiebarometeret suggests substantial national variation in the teacher led learning, ranging roughly from 7 to 11 hours per week, as well as in the number of hours that a Norwegian sociology student invest in self-study, ranging roughly from 11 to 23 hours per week. It is not reasonable that students within essentially the same disciplinary programme receive over 50 % more organised teaching in one university than in another. It is also not reasonable that the student workload can vary more than 100% between universities for what is basically supposed to be the same education. If anything, one would expect that the highly selected and academically strong students at UiO and NTNU would have to invest less time into their sociology studies than less selected and weaker students at some local universities. Nevertheless, the situation is the reverse. This discrepancy is a strong indication that the learning requirements put on students depend not so much upon which programme they choose, but far too much upon which university they attend. The panel considers this to be a serious quality issue for the system as a whole.

    General remarks, internationalisation
    Internationalisation is a broad concept that encompasses various dimensions and learning experiences. Most programmes (BA, MA, PhD) offer their students opportunities to study abroad. The degree to which programmes facilitate such foreign exchanges differs, however, and it is important to overcome barriers wherever possible (e.g., financial costs, poor institutional support) that prevent students from actually going abroad. There are some exemplary programmes that have extensive agreements with multiple universities abroad, and that actively support their students to make use of these possibilities. Internationalisation is more than ‘sending students abroad’, however, and also includes having an international outlook within the courses of the programme. At a minimum, according to the panel, this implies that teaching materials adopt an international perspective. Currently, however, there appears to be a tendency in most sociology programmes to focus on Norway, i.e., most (text)books and articles are in Norwegian, many are originally written by Norwegian authors, and the course topics are often strongly focusing on Norway. The panel invites these programmes to adopt a more global perspective in the teaching materials, to include more of the international (English) state-of-the-art literature, and to teach students the merits of cross-national, comparative research (e.g., in terms of social policy and best societal practices). Another aspect of internationalisation is having an international classroom, which can be a great learning experience for (foreign and Norwegian) students. Some programmes succeed in creating such an international environment, by offering courses in English and having a website and course descriptions in English, but many more do not. Offering courses in English should, needless to say, not be a goal in itself, and hence the panel is sceptical about the value of English courses which are not attended by Norwegian students. The panel is also sympathetic to some programmes that invest less heavily in international student exchange, when the staff of the programme is small, and/or when the aim of the programme is to attract students only locally and make them ready for jobs in the local labour market.

    General remarks, relevance
    Sociology programmes in Norway aim to teach students to think critically and independently, and this is what programmes accomplish very well. This is general competence, and students tend to be very satisfied with this. There is a need in most institutions, however, to develop the relevance of the programme for future employment. There is also a need for learning outcomes to more consistently provide a language for students and employers to articulate the skills that sociology programmes provide and that are valuable for the labour market. Put simply: if we can’t say it, students can’t say it. UiO’s careers website provides an example of best practice in the way in which it discusses and presents sociology careers, and the schemes it employs, e.g., ‘shadow a sociologist’ and ‘work placements’.

    The majority of programmes would also benefit from developing occupational and professional skills through consistent, cumulative research methods training. The research skills of sociology students are highly valued in the labour market, both in more research-oriented occupations, and for social policy jobs.

    Relevance can also be improved through revisions to programme content. UiB provides an example of best practice in showing how revisions to the content and approach of sociology programmes are positive for enhancing the relevance of study. This includes efforts to develop theory teaching away from a history of ideas and towards sociological analysis. Other ways to enhance relevance through content is by increasing international and comparative perspectives.

    General remarks, educational leadership
    Most programmes have appropriate management structures and mechanisms for programme monitoring in place, including internal and external evaluations of programmes. Student democracy through involvement in educational leadership committees is generally very good. Across the majority of institutions, the panel found that different levels are taking part in the development of the teaching programmes (i.e., director, staff, students). There was less evidence, however, of strategic leadership over programmes and the effectiveness of leadership in securing appropriate resources for programmes. Many programmes encountered problems over resource allocation (short on staff, appropriate teaching rooms, finances and so on) and it is here that effective sociology-specific leadership strategies would be most valuable.

    Read the report (pdf)