EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT PROGRAMME ON PUPIL WELL-BEING IN PRIMARY SCHOOLS

This study evaluates the effectiveness of a large-scale training programme on pupil well-being. The research questions are (1) to what extent did the training programme have an impact at the level of schools, and (2) to what extent did the training programme have an impact at the level of pupils? Using a survey with teachers and school principals (n=899), a survey with primary school pupils (n=2,612), and semi-structured interviews and focus groups with the main stakeholders (n=14), it was concluded that, notwithstanding high participant satisfaction, no substantial effects on school practices or on pupil well-being could be observed. Insufficient training time and the absence of post-training school counselling were key factors in the lack of success.

Many studies have shown that teachers and schools can have an impact on the well-being of pupils (e.g.Suldo, Friedrich, White, Farmer, Minch & Michalowski (2009)).Therefore, in Flanders -just as in other regions and countries -several projects and training programmes have been initiated by schools, counselling services, and authorities in order to support the schools' and teachers' capacities to strengthen pupil well-being.The evaluation of those initiatives is mostly limited to mapping participants' initial impressions of the training sessions, which almost without exception results in the conclusion that participants were satisfied (Guskey, 2002).Evaluations only seldom include impact measurement.It is thus assumed that these training programmes are successful, even though no evaluation has been conducted that goes beyond the level of immediate participant satisfaction (Mathison, 1992;Rossi, Lipsey & Freeman, 2004).
A number of studies have investigated the effectiveness of training or intervention programmes on well-being, on the reduction of bullying, and on mental health.Positive effects on the reduction of bullying were found by Olweus (1993), Evers, Prochaska, Van Marter, Johnson, and Prochaska (2007), Kyriakides, Creembers, Muijs, Rekers-Mombarg, Papastylianou, Van Petegem, and Pearson (2013), Eslea and Smith (1998) and Slee and Mohyla (2007), but some approaches or interventions were less effective or only effective for some groups of pupils, as shown by Rigby and Slee (2008), Smith, Schneider, Smith, and Ananiadou (2004), Stevens, Van Oost, and De Bourdeaudhuij (2001) and Stevens, De Bourdeaudhuij, and Van Oost (2000).A review by Merell, Gueldner, Ross, and Isava (2008) points out that there is often an effect on pupil or teacher knowledge, but only rarely on actual well-being or bullying behaviour.There is currently no clear view on the features of training programmes that could explain their success or lack EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT... thereof, due to the absence of a clear conceptualization of well-being as well as to the wide variety of evaluation methods used to evaluate training programmes (Hascher, 2008;Stevens et al., 2001).
This article reports on an effectiveness study based on the quasiexperimental approach of a training programme which has as its primary aim the enhancement of primary schools' capacities to strengthen the well-being of pupils.The programme was initiated by an educational authority in collaboration with three in-service training organisations.From the programme's start in 2004, teachers and school principals of 115 primary schools were trained to strengthen the well-being of both pupils and teachers.Participating schools were free to select one to four staff members to join in the training sessions, together with the school principal.Those participants were tasked to train and involve the colleagues who did not participate directly in the training.Among the main issues of the programme's training content is the idea that enhancing well-being is, in line with current views on change management, effective in schools when the well-being is enhanced from an educationally holistic point of view ('whole school approach'), when the process starts from a comprehensive analysis of the current situation in schools (rather than based on temporary or suddenly arising priorities or problems), and when the focus is on promoting a child-friendly school climate rather than on topic-related prevention or problem solving.Schools that have entered the programme were trained in three-day sessions mostly spread over the course of three to five months.During the first day of training, general ideas were highlighted (e.g. the conceptualization of well-being, the need for a child-friendly school climate, and the importance of whole-school initiatives).On the second day, the participants exchanged ideas on how to promote a child-friendly school climate, and participants were provided with a framework for school self-analysis which allowed them to assess their current strengths and weaknesses regarding pupil well-being.On the third training day, participants drew up a plan suited to the needs of their school.Participating schools were required conduct a comprehensive analysis and to develop a school-specific path based on the assessment results focused on the well-being of the pupils.The concrete initiatives to be undertaken in order to strengthen pupil well-being were not pre-aligned as they were the result of the school self-analysis.Actions undertaken by the schools thus differed widely between schools.
In the scope of this article we discuss two research questions, namely: 1.To what extent did the training programme result in effects at the level of schools? 2. To what extent did the training programme result in effects at the level of individual pupils?

Theoretical framework
In this part of the article, we discuss the evaluation levels we used to measure the effects of the training programme.We also provide definitions for the concepts of 'well-being' and 'schools' policy-making capacities'.
The predominant aim of the present study was to evaluate the effectiveness of the training programme described above.Several kinds of effects were measured using the four levels of the Kirkpatrick model (Kirkpatrick, 1998): 'reaction', 'learning', 'behaviour', and 'results'.We refer to Table 1 for more details on those levels, as well as for their application in this research.The four stages are consecutive: the effect of a certain stage can be reached only when the effect of each of the previous stages has been achieved.

Description evaluation level
Applied in the present study

Reaction
The degree to which participants react favourably immediately after the training programme.
The satisfaction and involvement of participants reported after each training session.

Learning
The degree to which participants have learned the new knowledge and skills as planned by the objectives put forward.
The learning effects on participants (knowledge and skills regarding enhancement of well-being).We also investigated the learning effects on the colleagues of participants.

Behaviour
The degree to which participants and schools apply what they have learned in daily practice and policy.
The effects of the training programme at the level of policy and practice of the school: did the learning effect result in changes in the school behaviour?

Results
The degree to which targeted outcomes occur as a result of participation in the training programme.
The effects of the training programme at the level of well-being of individual pupils: did the changes in school behaviour result in the increased well-being of pupils?
We needed a clear definition of 'pupil well-being' in order to measure the effects of the training programme at level 4 ('Results').In the academic field, well-being has been conceptualized in very different ways, from very narrow to comprehensive interpretations.A literature review by Hird (2003) concluded that there is no general agreement on a universal definition of pupil well-being.We have opted for a broad definition, often used in the context of Flemish education (De Volder & De Lee, 2009).We used this definition because it is based on a large-scale literature review and empiric validation, it is easily applicable in the Flemish education context, and it is intended to encompass all possible definitions.Consequently this conceptualization of well-being is rather broad.Well-being is "how a child experiences the school based on his current and his sustainable school experiences.The concept is multi-dimensional and is the result of the child's perception of the dimensions of satisfaction, involvedness in learning activities, academic self-concept, social relations and educational climate" (De Volder & De Lee, 2009).The five dimensions identified by this definition, each contributing to the overarching concept of pupil well-being, are defined in turn in Table 2.We used the concept of policy-making capacities in order to study the effects of the training programme at the level of the policy and behaviour of the school (level 3, 'Behaviour').Policy-making capacities of schools are described as "the extent to which schools use the available room for policy-making to come to a continuous process of retaining or changing their work in order to improve their educational quality and attain both the external and self-imposed objectives" (Vanhoof, Van Petegem, Verhoeven & Buvens, 2009).The concept of policy-making capacities consists of eight empirically supported indicators.Each of those indicators were focused on during the training programme: orientation towards shared goals, effective communication, involvement by shared leadership, integration of different policy initiatives, orientation towards innovation, supportive relationships and cooperation, reflective capacity, and responsiveness towards expectations.
EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT...

Orientation towards shared goals
The extent to which actions are embedded in a vision supported by all stakeholders.

Effective communication
The extent to which both formal and informal communication meet the needs for information and the needs for reciprocal influence and alignment.

Involvement by shared leadership
The extent to which the school staff actively participates in decision-making processes.

Integration of different policy initiatives
The degree of alignment of different individual policy areas within the broader school policy.

Orientation towards innovation
The extent to which the school is open to innovations, copes with changes, and is able to implement innovations successfully.

Supportive relationships and cooperation
The extent to which professional and personal relations between school staff are experienced as supportive.

Reflective capacity
The extent to which the school takes the initiative to identify strengths and weaknesses in its own performance.

Responsiveness towards expectations
The extent to which the school is open to, and is able to respond to, external demands and expectations.

Research method: A quasi-experimental multi-method design
A quasi-experimental research approach was set up to compare the policymaking capacities, current approaches regarding well-being, and the actual well-being of pupils of the schools that participated in the programme (experimental group), with those of schools that did not participate in the programme (control group).The label 'quasi-experimental approach' is used, as schools could not be assigned at random to either the experimental or the control group.Data within the experiment were collected using a parallel multi-method design.A combination of inductive and deductive research logic was used (Tashakkori & Teddlie, 1998).This choice was dictated by the value of an appropriate concurrent combination of different research methods (Creswell, Clark, Gutmann & Hanson, 2003).It guarantees deeper and broader answers on the formulated research questions.Quantitative data can indicate the relative impact of a particular variable and indicate the level of effects, for instance by using statistical analysis.Qualitative data make it possible to understand how actors in the field speak about a studied phenomenon.One method may offset the defects of another.The essential aim is to produce data triangulation and methodological triangulation.This means that the findings of different methods of analysis are integrated into the analysis (Miles & Huberman, 1994).The (generalizable) findings from survey research are compared with the results of example-inclusive in-depth interviews (Creswell, 2003).

Data collection
Three groups of data were collected: feedback sheets immediately after the training sessions, survey data with school staff and pupils, and qualitative data consisting of in-depth interviews and focus groups.

Feedback sheets
Data on the effects at level 1 ('Reaction') were deducted from an analysis of feedback sheets filled in by the participants immediately after the training session.In total, 336 feedback sheets were collected by the trainers, divided over the three training sessions.Next to an ordinal scale from 1 to 5 on which participants indicated their involvement during the training activities, a few open questions were asked.The involvement feedback sheets also contained data that were useful for measuring the effects at level 2 ('Learning').

Quantitative survey with school staff and pupils
Survey with school staff.Online questionnaires were administered to 899 teachers, school principals, and other educational staff (overall response rate of 68.2%) of a representative sample of 66 primary schools.The quasi-experimental group consisted of 513 school principals and staff from 36 schools that had attended the training; for the control group, we collected data from 386 school principals and staff from 30 similar schools.Both samples were stratified and are comparable in terms of school size, educational network, and percentage of pupils with low socio-economic status.The group of respondents consisted mainly of classroom teachers (78.4%), but also principals (5.0%), 'guidance coordinators' or 'SES-teachers' (appointed to promoted equal opportunities) (10.6%), and other educational staff (6.0%).The sample included staff with a variety of teaching experience.
Survey with pupils.In our sample of 66 primary schools, we tested the wellbeing of pupils in grades 7 and 8 (ages 12-14).From each of those schools, one class of grade 7 pupils and one class of grade 8 pupils was selected at random.In all, 2,612 pupils completed a written questionnaire containing 28 Likert-type scales, answering categories from 1 to 4. The response group was evenly spread between boys (49.4%) and girls (50.6%), and between grades (48.1% from grade 7; 51.9% from grade 8).The quasi-experimental group consisted of 1,536 pupils (in 36 schools); 1,078 pupils were in the control group (30 schools).
EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT...

Qualitative study: in-depth interviews and focus groups meetings
In the parallel qualitative part of this study, we conducted semi-structured in-depth interviews with 14 stakeholders of the training programme.The stakeholders were school principals (n=3) and staff (n=3) from participating schools, school counsellors (n=3), staff, and policy makers involved in the organisation of the training programme (n=3), and network partners that supported the training programme (n=2).The primary aim of these semistructured in-depth interviews was to obtain an enriching and reliable deeper understanding of points of view and expectations regarding the effects of the training programme and to find out which opinions were based on features that facilitated or obstructed the success of the training programme.The choice of in-depth interviews was supported by the added value of this technique in exhibiting multiple facets from different points of view (Creswell, 2003;Tracy, 2010).The interview data were enriched by two focus group interviews consisting of 5 stakeholders selected from the 14 respondents interviewed in the previous phase.The aim of the focus groups was to discuss and reflect on the preliminary results.In contrast with the in-depth-interviews, the stakeholders joined in one of two conferences allowing the respondents to react to each other's comments.The interactivity of this method, in which the moderator facilitates rather than directs the conversation, allows reactions and interpretations to be captured and confusing or adverse data to be clarified (Kitzinger, 1994;Liamputtong, 2010).The qualitative research data was transcribed, coded based on the conceptual framework, and analysed using Nvivo.

Instruments and analysis
The well-being questionnaire was developed by De Volder and De Lee (2009).It was administered by the classroom teacher or the school principal under strict guidelines described by the research team in order to ensure the reliability of the test.The test measures five dimensions that are part of wellbeing, as defined in the theoretical framework: satisfaction, involvedness in learning activities, academic self-concept, social relations, and educational climate.An indicator for each dimension and information on the internal consistency of the scales are provided in Table 4. Educational climate (Cronbach's α=0.80) The teachers listen to the opinions of pupils.
An existing instrument (Vanhoof et al., 2011) was adapted in order to measure the policy-making capacities regarding the well-being of pupils.Table 6 presents information on the internal consistency of the scales used.In analysing the survey data, we made use of t-tests (and effects sizes) to answer the research questions concerning the differences between the experimental and control schools.Most of the time only descriptive results are presented, given that no engagement in inferential statistics is needed to answer the research questions.

Results
As elaborated above, the present study was set up to gain information on four levels of effect as described by Kirkpatrick (1998) (see the section on 'theoretical framework'): (1) reaction, (2) learning, (3) behaviour and (4) results.Whereas 'reaction' measures the initial opinions of the participants immediately after the training, the subsequent stages concern the learning effects on participants (and their colleagues), the transfer to teacher and school behaviour in daily practice, and the impact the training has on wellbeing as experienced by the pupils.
Level 1. Reaction An analysis of the feedback sheets (n=336) shows that the training programme results in a high level of satisfaction among participants.On a scale of involvement from 1 to 5, a vast majority of the participants (237 out of 336; 70.5%) scored their involvement during the training session as a 4, indicating they agreed with the statement During the training session I was quite often or regularly fascinated or touched.At these moments, I was motivated to think about or do something with the presented content.Additionally, 12.8% of the participants gave the maximum score of 5.
EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT...Among the features spontaneously highlighted by participants as contributing to satisfaction were the coaching style of the trainer (mentioned on 27.7% of the involvement feedback sheets), the training methods (28.8%), and the exchanges with colleagues from other schools (18.5%).The organisational and practical aspects were also mentioned as important factors for participant satisfaction (29.5%).

Level 2. Learning
The feedback sheets (n=336) filled in by the participants immediately after the training programme showed that the participants had acquired a wide array of new knowledge and skills.Of the comments on the learning effects, 74.7% were positive statements.On 37 involvement feedback sheets (11.1%), participants stated that they did not learn much during that session; this was mostly the case after the first session (32 of the 37 times this feedback was given).Other feedback included the need of participants to receive more concrete solutions for very specific problems, as well as their need for more practice-oriented information.Both kinds of feedback were filled in on 27 involvement feedback sheets (8.0%) and mainly concerned the first -most theory-oriented -session.It can be assumed that those expectations were met during the second or third training day, as the number of negative comments declined with each session.
The strong learning effect was confirmed by an analysis of the quantitative data from the questionnaire for school staff members (section for participants in the training programme, n=88) which revealed that 84.1% of the participants (the sum of the positive categories 'strongly agree' and 'agree') had a clear view of the basic principles of the training programme.The participants also indicated having acquired new knowledge (82.1%) and new skills (83.0%).For more detailed results, see Table 5.The learning effect of non-participating colleagues was estimated as significantly lower: only 53.6% of the participants judged that their colleagues knew the basic principles of the training programme.The higher standard deviation in this item indicates that answers are less consistent.This finding is problematic for the effectiveness of the training programme, as the involvement of all staff members is perceived as a crucial factor for the successful implementation of the training programme in the school's policy and practice.
Level 3. Behaviour As described in the methodology section, data on the effects at level 3 were collected through a quasi-experimental approach with a questionnaire for school staff members (n=899).An analysis of the questionnaire data indicated that the training programme had no considerable effect on the policy of schools in terms of policy-making capacities regarding well-being.Although the training programme was intended to influence and strengthen the school policies regarding well-being, for seven of the eight indicators of policy-making capacities we failed to find a statistically significant difference between the data of the staff of the quasi-experimental group and the staff of the control group.The only indicator that showed a statistically significant difference was 'effective communication' for which the non-participating schools obtained a slightly higher score on a Likert-style scale from 1 to 4 (t=-2.04,df=883, p<0.05).However, Cohen's d of 0.13 indicates the effect was negligible.The questionnaire contained 11 items for both the staff from quasiexperimental group and the control group to discern in which matters differences occurred.These items measured specific outcomes expected by the training programme developers, e.g."We pay a lot of attention to the well-being of pupils', 'I think our school is working to enhance the well-being of pupils in a qualitatively good manner', and 'I think our school is working to enhance the well-being of pupils in a more qualitative manner than we did five years ago."In line with the findings related to the policy-making capacities, no statistically significant differences were observed on those items between the two groups of schools.Another noteworthy finding was that the school staff in schools that had entered the program and in schools that had not both estimated that more attention is being paid to the well-being of pupils than five years ago.
The limited effect of the training programme on the behaviour of schools was confirmed during the in-depth interviews by 7 out of the 9 respondents that had participated in the training programme.There were two exceptions: one interviewed school principal stated that the training programme was the catalyst for her school to start an encompassing analysis of the school which lead to concrete initiatives for improvement.Another positive story came from a school counsellor who set up an intensive coaching program together with the school principal and her board, which would result in a major shift in the school's handling of pupil and teacher well-being.These effects were only obtained on a small scale, as the quantitative part of this study showed no significant effects.
The absence of robust results at the level of 'behaviour' was the most important topic during the two focus groups.All the respondents agreed that the training time was too short to enable schools to build policy-making capacities regarding well-being.The respondents mentioned that one of the most important factors that determine whether schools can turn the learning effect into changed practices and policies is the extent of the policy-making capacities in the schools before their participation in the training.All of the respondents agreed that more intensive post-training treatment is a necessary condition for strengthening the effects at the level of 'behaviour'.
A factor mentioned by the respondents during the focus groups which is likely to have negatively affected the results is that not all teachers are truly convinced of the importance of well-being, even in the participating schools.One of the training programme's operating assumptions was that every team member would be willing to get involved in the implementation.Another factor perceived by all respondents was the high experience workload of teachers and school principals in Flemish education.Finally, according to the focus group respondents, schools tend to focus on short-term gains, which obstructs attention for improvements that require more time.

Level 4. Results
The questionnaire for school staff members (the section only for participants in the training programme, n=88) showed that a majority of the participants (51.3%) 'agrees' with the item 'the well-being of pupils was enhanced as a result of the school entering the training programme', with an additional 11.8% who 'strongly agree'.The questionnaire for staff members includes the items 'in our school pupil wellbeing is high' and 'in our school pupil well-being is higher than it used to be five years ago'.There were high scores on both items (93.6% and 76.6% of the respondents 'agree' or 'strongly agree') in the quasi-experimental group, but comparably high scores were obtained in the control group of non-participating schools (p=0.23 and p=0.18, the latter result the opposite of what had been expected).
The questionnaire for pupils (n=2612) measured the well-being of pupils in five dimensions (see section on 'theoretical framework').After calculating Cronbach's alpha for each of the dimensions, one item of 'satisfaction' and one item of 'involvedness in learning activities' was eliminated from further analysis.This resulted in a satisfactory internal consistency of the scales used (see Table 5).The pupil questionnaire exposed hardly any difference between the pupils of schools from the quasi-experimental group and the pupils of the control group.A slightly negative effect was even found for the dimension 'social relations' (t=3.30;df=2610; p=0.001), but a Cohen's d score of 0.13 indicated that the effect was very small.None of the data on the other dimensions revealed statistically significant differences.In spite of participants' strong belief in the positive effect of the training programme on the wellbeing of pupils, these limited effects were to be expected given the small effects at the level of school behaviour (see the paragraph on 'level 3').

Discussion
In the present study, we investigated the effectiveness of a training programme on well-being in primary schools.The programme obtained encouraging results at the 'lower' levels of the Kirkpatrick model: there was a favourable reaction immediately after the training and a high learning effect reported by participants.However, the learning effect did not significantly affect the schools' policy-making capacities regarding well-being.We also did not find statistically significant differences between the well-being of pupils from participating versus non-participating schools.
In the remainder of this article, we identify a number of key factors that might help explain the absence of success of the training programme.We also discuss the paradoxical role of policy-making capacities in the success of the training programme and finally the training programme's focus at the school level rather than at the teacher level is questioned.

Explaining the absence of effects of the training
The assertion from the focus groups that a higher training intensity and a longer engagement in time are necessary factors in order to obtain substantial success at the 'behaviour' and 'impact' levels has been confirmed by several studies.A review by Wells, Barlow, and Stewart-Brown (2003) reveals three key factors that influence the success of programmes on well-being and mental health in primary schools: 1) school-based programmes (rather than single classroom interventions) supported by all team members, aimed at changing the school's culture; 2) programmes that continue over an extensive period of time (one year or longer), since interventions that aspire to make fundamental changes in a short-time period (such as the training programme that was subject of this study) are rarely effective; and 3) programmes that focus on mental health promotion rather than on preventing mental illnesses are more likely to obtain robust results (Wells et al., 2003).While the training programme subject of this study seemed to meet the first and last condition mentioned by (Wells et al., 2003), the training time was relatively short (three days, most often spread over four months) and hardly any post-intervention treatment was organized.Wells et al. (2003) confirmed that schools may require support during the implementation phase, as changes in adult behaviour are difficult to achieve.The timing of interventions and the availability of support is found to positively affect the sustainability of change initiatives (Buchanan, Fitzgerald, Ketley, Gollop, Louise Jones, Saint Lamont … & Whitby, 2005).Necessary conditions for successful intervention programmes in the field of pupil well-being identified in a review by Green, Howes, Waters, Maher, and Oberklaid (2005) are a relatively stable and sustainable focus on pupil well-being and self-confidence and an integrated MAARTEN PENNINCKX, JAN VANHOOF, PETER VAN PETEGEM approach at the school level.According to a review by van Veen, Zwart, Meirink, and Verloop (2010) several studies suggest that the availability of 'a substantial amount of time' is the most crucial factor in the success of training programmes in this area.In contradiction of these findings, Evers et al. (2007) found a reduction in bullying (in middle schools and in high schools) as a result of a short-term intervention.This latter study concluded that interventions should be organized with a relatively low intensity, as the barriers to dissemination and successful implementation in schools are larger in programmes with greater intensity and higher demands on the school staff (Evers et al., 2007).In addition to extensive time and intensive post-training treatment, the focus group respondents identified three more factors that could have affected the results negatively.There was general agreement that not all teachers are truly convinced of the importance of well-being, while involvement of all staff is a requirement in the holistic view of the training programme.Furthermore, the focus group respondents argued that many teachers and other school practitioners already think their workload is too high.The implementation of the training programme requires a strong commitment in terms of resources and time, while most practitioners are mainly focused on daily and direct tasks with short-term gains and have less time for the implementation of an intensive programme.Flemish schools' focus on short-term tasks and daily management rather than on long-term policies was also observed by a recent OECD study (Shewbridge, Hulshof, Nusche & Stoll, 2011).

A paradox regarding policy-making capacities
The focus group respondents agreed that the importance of the policy-making capacities of schools cannot be underestimated in present-day education.Therefore the training's primary intention to strengthen schools at the policy level was widely supported.Although this is slowly changing in present-day Flemish education, a considerable number of schools still lack strong policymaking capacities and need a long trajectory of coaching and reflection to further develop these capacities (Ministry of Education and Training, 2010;Shewbridge et al., 2011).The few success stories of the training programme that were mentioned during the in-depth interviews and the focus groups suggest that the training programme could only be implemented successfully in a school where participation in the training programme was already thoroughly discussed with members of the school team leading to a shared vision and expectation (suggesting the school had already developed strong policy-making capacities beforehand), and in a school that enjoys comprehensive and intensive multi-year coaching from an external consultant.This finding marks a paradoxical situation: policy-making capacities before the training programme are a necessary (but insufficient) condition for strengthening EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT... the schools' policy-making capacities further.Schools that had weak policymaking capacities were not able to put into practice what they had learned from the training programme.

Recommendations for research and practice
The training programme was intended to have an impact on pupil well-being by strengthening capacities at the school level.However, Opdenakker and Van Damme (2000) assert that the effects of school characteristics on wellbeing are rather limited.Their research showed the main effects on the well-being of pupils were through 'teaching staff co-operation in relation to teaching methods and pupil counselling' and 'an orderly learning environment'.According to these authors, training programmes on well-being therefore should focus mainly on those school features.Van Landegem and Van Damme (2002) identified mainly teacher characteristics as the explanation for declining well-being throughout a pupil's studies.This study found that the pupils are more likely to experience high well-being in a school with teachers who have positive feelings about their work, who are able to get pupils actively involved during learning activities, and who are aiming at the holistic development of pupils.Gutman and Feinstein (2008) found that only 3% of the variation in a pupil's mental health is explained by school characteristics, while the pupil's well-being is more dependent on the individual experiences of the children in schools such as their individual interactions with peers and teachers.This was confirmed by the research of Murray-Harvey and Slee (2010), which highlights the importance of teacher-pupil and peer relations in the reduction of bullying.Suldo et al. (2009) found that perceived teacher support accounts for 16% of variation in student well-being.
Nevertheless, studies by Kyriakides (2005) and Kyriakides et al. (2013) and reported a substantial effect of school characteristics on pupil well-being.According to Zullig, Huebner, and Patton (2011), the most influential school characteristics on well-being are the level of academic support and the extent to which the pupil-teacher relationship is perceived as positive.This study also observed positive correlations between well-being and school connectedness, order, and discipline, the physical and the social environment of the school, and academic satisfaction, and a negative correlation with perceived exclusion/ privileges.Finally, Saarento, Kärnä, Hodges, and Salmivalli (2013) found that although most of the variation regarding victimization is due to differences among students, substantial variance also exists among both schools and classrooms.It should be noted that one cannot easily compare the various studies on this subject because of the wide variation in the conceptualization of well-being and in the methodologies used to measure well-being (Hascher, 2008;Hird, 2003).The question of whether variance in pupil well-being is to be explained mainly by school or teacher characteristics currently remains largely unanswered.
Because of the absence of tangible results at levels 3 and 4, consistent with the literature described above, we argue that it is advisory for developers of training programmes on well-being in schools to include or to support post-training coaching initiatives, and that a clear path of post-training school counselling should be presented to schools.Furthermore, we advise creating opportunities for schools to call on the services of expert help, to use an intake procedure to ensure that the main stakeholders of the participating schools (at least the school principals and the school staff ) are willing to invest time and resources in the implementation of the training programme, and to focus on those school characteristics that have proven to have a main effect on well-being of pupils.Evidence from research needs to contribute to the training programme content in order to ensure a transfer from level 3 (behaviour) to level 4 (results).The present study also serves as a basis for advocating more research on key factors of schools that effectively have an impact on the well-being of pupils and on key factors for the effectiveness of in-service training and coaching related to pupil well-being.For this purpose, a clear conceptualisation of 'well-being' is required.
Our study presents evidence that high participant satisfaction immediately after training programmes is not a good predictor for their impact.Therefore we advise developers of training programmes on well-being in schools to reflect on ways to create a culture of permanent evaluation of the training programme's effects, including baseline studies.Programmes that include systematic progress monitoring are more likely to achieve success in implementation (Smith et al., 2004).

Table 2
De Volder and De Lee (2009)ng as described byDe Volder and De Lee (2009)

Table 4
Indicators and Cronbach's alpha for the well-being questionnaire

Table 5
Self-reported learning effects on participants in the training programme

Table 6
Indicators of policy-making capacities by participation in the training programme EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT...

Table 7
Dimensions of pupil well-being by participation of the school in the training programme EVALUATING THE EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT...