Effectiveness of Schools’ Management and Its Measurement – a Review of Concepts

Purpose – Identification of factors determining the effectiveness of public schools’ management, and the establishment of means for developing the concept of measuring the schools’ effectiveness. Methodology – Review and assessment of concepts and studies present in Polish and international literature pertaining to the evaluation of public schools’ effectiveness. Findings – The hitherto methods for evaluating schools are unsatisfactory. It is largely associated with ambiguous definitions of schools’ effectiveness and identification of objectives and tasks schools are to achieve and undertake, associated with it. This, in turn, results from the fact that effectiveness assessment may by defined differently by various stakeholders. However, the measurement of effectiveness is necessary due to the fact that it facilitates the emergence of high quality educational services in Polish public schools. Research implications – The review of literature indicates that the assessment of educational added value over the course of several years of education seems valid. This means that rather than tracking final results achieved by students, the growth of students’ knowledge ought to be measured. The Data Envelopment Analysis (DEA) method may be helpful in relation to the above. The method seems well-suited for evaluating the effectiveness of NGO sector institutions, and takes non-financial factors into consideration, which will enable methodologically categorized knowledge on the effectiveness of schools’ management to be obtained. Practical implications – Results of the present study may be applied by institutions supervising public schools and by individuals who manage these institutions in order to perform periodic assessments of schools’ effectiveness, and who would stimulate the education quality improvement. Social implications – Results of the present study may foster the improvement of schools’ competitiveness and improvement of their educational actions’ quality. Originality/ Value – The paper supplements knowledge regarding the assessment of Polish public schools’ effectiveness. Type of article – A theoretical paper.


Introduction
The discussion on the improvement of educational institutions' effectiveness has been ongoing both in Poland and globally. Several countries developed programs aiming to collect school quality indexes in order to monitor national, regional and local results. Focus on the search for further, complex data has been widespread. Both teachers Pobrane z czasopisma International Journal of Synergy and Research http://ijsr.journals.umcs.pl Data: 06/06/2021 05:15:23 U M C S 106 IJSR 6 themselves and managers of educational institutions increase their efforts to collect and analyze such data in order to accurately describe the operation of education systems. At present, virtually each European country has applied a system for monitoring educational processes based upon efficiency indexes. Frequently, focus upon monitoring effectiveness becomes the objective in itself instead of a method for improving the actual effectiveness of education systems. Trujillo (2014) labels this as "the new cult of effectiveness". The author argues that the introduction of changes in the education system based solely upon the available data creates "a new technocracy" and basically does not solve many significant problems in public education, which, in addition, are usually difficult to be measured. On the other hand, Klein (2017) indicates that the current discussion regarding effectiveness management is usually focused upon issues such as the frequency of tests and the time they are given at. Others, such as Mitchell (2017), observe that effectiveness frequently denotes the level of proficiency, i.e. the ability to solve tests. According to Sawchuk (2013), it pertains to the way results are calculated, which, at the same time, offers the basis for assessing the quality of teachers' work. Another problem associated with the assessment of effectiveness can be raised. It pertains to whether similar standards ought to be enforced for all student groups, and which types of activities schools ought to be responsible for (Hoff, 2007).
It ought to be emphasized that all issues indicated by researchers are organizational and process-related in character. However, they are not linked with any significant problems the public education must face at present. Issues pertaining to what students ought to know and be able to do, which schools ought to be publically supported, how to measure other objectives of public education apart from the academic knowledge, who ought to supervise and control decision making regarding educational policies, are usually discussed much more rarely. In addition, effectiveness indexes applied so far have not offered conclusive answers.
Effectiveness management systems in educational institutions are relatively new in their current forms. However, the idea of "creating a single, best system" is not a novel concept (Tyack, 1974). The discussion on how to measure the outcomes and how to translate these into specific educational initiatives has been ongoing for years and stimulated a variety of political initiatives. In light of the current educational reform, the issue of effectiveness management constitutes a current and crucial topic.

Effectiveness in education management
The assessment of effectiveness, as an element of education management, constitutes one of the vital elements of evaluating the operation of schools and educational institutions. Such an assessment results from legal, market and methodological regulations. The obligation of analyzing and assessing the outcomes of didactic, educational, carerelated and other statutory activities of schools and institutions is regulated in the Act on the Education System (1991 as amended).
The assessment of schools' operations constitutes an element of pedagogical supervision and is to determine the degree to which schools meet legal requirements set by the state. The act introduced a hierarchical supervision: school director, superin- tendent of education, the minister of education. Monitoring visits generally result in a summary report being drafted. Rankings of schools published by local and national media constitute a response to the demand for straightforward information regarding the position of a particular school in relation to other institutions. The rankings take the form of a single-criteria (rarely several-criteria) compilation composed on the basis of arbitrarily selected indexes and arbitrarily set weight. The average exam score achieved by students constitutes the most frequent index. According to employees of education sector, these results do not offer ample grounds for composing rankings. The assessment of schools' operation based solely upon examination results often presents their work in false light. Evaluating schools' efficiency only in light of the results of examinations, without conditions the institution functions in being taken into consideration, is ungrounded and may be unfair or unfairly positive. Mere results of examinations, with no social and educational contexts, ought not to constitute the only measure of quality.
The term "effectiveness" should be defined before a deeper analysis of the issue of public educational institutions' management is conducted. In accordance with the organizational theory featured in management studies, three approaches to effectiveness can be indicated: • objective-based approach -the assessment of effectiveness pertains primarily to the degree the targeted objective is achieved to. The level the available resources are utilized to constitutes a further factor in the evaluation, • systemic approach -effectiveness constitutes the organization's ability to overcome the lack of self-confidence emerging from its surrounding environment and the organization's shaping its environment in the direction it sees fit, • multi-criteria approach -the idea of effectiveness is based upon the organization achieving its objectives, meeting specific conditions and maintaining required standards (Pawłowski, 2004). In light of these issues, the term "effectiveness" pertains to the assessment of obtained benefits and the degree available resources are utilized to (Cooper et al., 2000, p. 66). Obviously, such an approach requires outcomes of educational processes and studies to be quantified, and all resources utilized in these processes identified. This may be based upon the definition offered by Pareto, which states: "the allocation of resources is effective when nobody's situation can be improved without the situation of someone else being aggravated" (Stiglitz, 2004).
"The effectiveness of education systems, and educational institutions which constitute their building block, is difficult to be evaluated. This is primarily due to the character of the object (knowledge) being processed" (Morawski, 1999, p. 135). The following constitute indexes of economic effectiveness applied in practice: education unit cost, financial result, or indirectly, an index constituting media value of the actual time the education runs for (Morawski, 1999, p. 151). The assessment of effectiveness requires a proper system of collecting and processing data. However, at present, in Poland and elsewhere, data originating from schools' periodic reports or statistical data published by the National Statistical Office are used in order to develop indexes. Therefore, apart from the development of these, the creation of information systems adjusted to the need emerging from the analysis of educational processes present in schools, is equally significant (Szuwarzyński, 2006, p. 80 The role of the assessment is highlighted by several modern management experts. Kaplan and Norton (2001) argue: "the assessment is significant because if something cannot be evaluated it cannot be managed". Pfeffer and Sutton (2002) observe: "what undergoes assessment is considered significant. Results of assessments exert an impact upon what people do, perceive and ignore. As a consequence, it is common knowledge that what is evaluated is usually applied. On the other hand, what is not assessed is frequently regarded as irrelevant. The significance of assessments is huge. The fact that they exert a considerable impact upon behaviors was acknowledged".
The above quotations clearly emphasize that in order to develop effective management tools, it is necessary to design indexes of effectiveness associated with processes present in schools. The evaluation of indexes such as education unit cost or financial results is relatively easy. However, the value of such an approach regarding the improvement of schools' effectiveness is arguable. As far as the development of indexes is concerned, problems associated with the proper definition of processes occurring in schools emerge.
Educational institutions must be considered as entities affected by several input and output variables, which is motivated by the multitude of processes realized by them. In many cases, the input/output criteria are problematic to measure. In addition, the application of economic criteria in the decision-making process is frequently impossible. The development of indexes describing schools' operations ought to be associated with e.g. the following strategic objectives (Al-Turki, Duffuaa, 2003, p. 331): education of highly qualified school-leavers, the development of graduates' awareness of the necessity for life-long learning, graduates' preparedness for team-working, graduates' preparedness for continuance of education, graduates' preparedness for entering labor market, provision of graduates' professional development in the course of life-long learning.
Such a brief, but by no means exhaustive, description of the issue indicates the complexity of the analysis regarding the effectiveness of processes realized in public schools. It may also constitute the basis for the development of tools facilitating the assessment of educational processes effectiveness conducted by managers of educational institutions.

Educational added value (EAV) -discussion of the issue
Dynamic changes occurring in the surrounding environment, the present education reform, and new challenges public schools face, require changes in the approach towards the effectiveness of educational processes. As a consequence, attempts are made to eliminate "raw results" as the sole index of educational quality, with the simultaneous implementation of Educational Added Value for evaluating the effectiveness of education.
Advocates of such changes (Dolata, 2007, p. 5) argue that EAV definitely constitutes a better alternative for raw exam results, which adjusts and revises the final exam outcome of the school by including the impact of factors the school has no effective influence upon, but which are indirectly included in educational outcomes. The EAV may be considered as an index of schools' work quality with regard to the scope tested Data Envelopment Analysis (DEA), universally applied for measuring effectiveness of NGO sector institutions, which cannot be characterized by indexes based upon financial factors, may be helpful in this respect. The method was developed in 1978 and was described as a mathematical programming model which supplies a new means for obtaining empirical assessments regarding the effective operation of organizations (Charnes et al., 1978). The empirical focus of the DEA, along with much simpler principles (in comparison with e.g. regression analysis), resulted in the method being universally applied in studies pertaining to the critical effectiveness of public, NGO and production sectors (Cooper et al., 2004, p. 2). The DEA method is a relatively new, data-focused approach to evaluating effectiveness of a defined set of objects labeled Decision Making Units (DMU) which are described by multi-inputs and multi-outputs (Cooper et al., 2004, p. 1). The DMU is understood as any object the effectiveness assessment is conducted for. As far as educational processes are concerned, the objects may be represented by individual schools. The following definition of effectiveness was developed for the purpose of the DEA: "full (100%) effectiveness is achieved by a DMU only when none of the inputs and outputs can be improved without impairing other inputs and outputs" (Cooper et al., 2004, p. 3). This fundamental type of effectiveness is defined as "technical effectiveness".
The fact that, when developing the effectiveness assessment model, a proper selection of inputs (resources utilized) and outputs (product processes) constitutes a significant problem. In general, a premise is made that graduates constitute learning process outcomes, curricula represent educational process outcomes, whereas inputs consist of all resources required in these processes (Al-Turki, Duffuaa, 2003, p. 331). By going beyond the analysis of a single outcome, the DEA method allows for several areas to be taken into account. Due to the fact that the method is a non-parametric one, no familiarity with the functional aspect of the input-output ratio is required. Both elements do not need to be expressed in monetary units and may be expressed in their natural value.
The application of the DEA method eliminates the issue of "market price" of individual inputs and outputs. In addition, the a priori establishment of weights of particular inputs and outputs is not required. The weight of individual elements is optimized so that the relative effectiveness of each object is maximized. Therefore, the DEA method's results may be considered objective. The existing extensions of the method enable the impact of the so-called environmental variables to be taken into consideration. These are the values an institution has no influence over (e.g. the ownership type or its geographical location) but which exert significant impact upon its work.
The literature of the subject dealing with education system management indicates that the traditional approach to evaluating educational institutions, based upon sole monitoring of their final achievements, entails specific adverse consequences. These frequently emerge when indexes measuring the percentage of students achieving scores which are higher than a particular threshold are utilized. This results in these institutions focusing upon pupils who are near the threshold and are likely to go beyond it. At the same time, teachers are demotivated from devoting their attention to talented pupils, who are nevertheless likely to achieve the required progress. Neal (2010) observes that, paradoxically, slower students may become the greatest losers in such a situation. This is due to the fact that teachers seek to avoid a defeat and devote their attention to students able to achieve higher scores. As a consequence, such an approach to outcomes' assessment neither reflects the actual situation nor exerts a suitable impact upon schools' operations. In addition, focus upon final outcomes does not take differences between individual students, who enter various schools, into account. As a consequence of such an approach, institutions working with low-achievers, or located in a disadvantageous social environment, may never have the opportunity to achieve a positive evaluation, which will be demotivating for their employees. The application of such an assessment approach is highly unlikely to offer equal educational opportunities.
Meanwhile, in order to improve the effectiveness of education system, the determination of schools' contribution to examination results seems of fundamental importance. EAV indexes allow to achieve, or at least come close to achieving, this objective. Importantly, the application of these indexes does not require the application of vertically aligned examinations (i.e. those whose results may be expressed on the same scale), which, as mentioned earlier, would be necessary to observe growth in the EAV ought to be applied in combination with other measures enabling other aspects of educational reality to be observed, especially with results of final exams. This is especially vital as far as grasping the problems of high EAV being accompanied by low final outcomes is concerned. Even though EAV indexes prove that low quality of schools' work is not always to blame for such state of affairs, these low results remain to pose a problem to be solved. The solution will undeniably require several initiatives to be undertaken, which would primarily target the environment the school operates in. On the other hand, when examining schools with low EAV, it can be said that from the system-related perspective (parents' perspective may be much different), schools combining low EAV with low outcomes are more troublesome that those where the EAV is low but final outcomes high. At the same time, it ought to be emphasized that the EAV ought not to be considered as the only fool-proof method for assessing schools' work quality, especially if the method is to serve as the basis for critical decisions, e.g. on salary bonuses, or the liquidation of institutions, to be undertaken (Żółtak, 2015, p. 11).
Even though, in Poland, the educational added value has only begun to gain popularity, it constitutes one of the significant indexes reflecting the quality of schools, and has been applied worldwide since the 1970s. Even though issues associated with the index are well-studied, the fact that it only examines a section of educational operations ought to be taken into consideration. Other indexes ought to be applied in order to give a full picture of educational institutions' effectiveness. The literature features opinions (Dolata, 2007, p. 14) stating that the EAV ought to be inspected in combination with examination results. The analysis of the EAV over the course of 3 years in combination with examination results covering the same period is suggested.
The educational added value enables the evaluation of educational effectiveness in the context of the best application of input resources, i.e. students' potential, teachers with their content-related and methodological training, the faculty board's capacity to undertake introspective initiatives, and school's management. The way these resources are utilized constitutes vital information for local governments, and allows actions addressing schools to be targeted. The establishment of schools' characteristics enables them to be managed more effectively. The EAV approach constitutes a tool of educational policy. It may be considered in the general context of education improvement policy via schools being evaluated upon their tangible outcomes. It may also be assessed in the context of problems emerging wherever educational sector market is created and whenever schools begin to compete with one another. The potential advantage emerging from the introduction of the EAV is seen in the frustration of good teachers working in difficult environments being diminished, and preventing less effective teachers transferring responsibility to the lack of students' motivation for learning and students' low intellectual potential.
Studies regarding the operation of education systems where schools compete for students and are financed on the basis of the number they were able to attract, indicate that, apart from obvious advantages of such an approach, there exist threats as well. The diversification of schools is indicated as the most frequent one. The weakest schools, from the point of view of final examination results, are not eliminated from the market, but for some reason vegetate with the number of pupils dwindling. Owing to the fact that students with the highest scores enter the strongest institutions, these consolidate their position on the market. Such a diversification results in the emergence of social segregation in education and poses a threat for a vital objective of educational policy, i.e. equal education opportunities.
The introduction of EAV indexes ought to be thought-through. The objectives they are to serve, and whether technical capacity (data availability) enables the satisfactory achievement of these objectives, ought to be analyzed. It ought to be noted that in practice, some defects of indexes, which emerge from, e.g. limited scope of control variables utilized in EAV models, may be mitigated by their suitable application in further analyses.

Conclusions
At present, Polish education system is undergoing deep changes. Several debates regarding the effectiveness of Polish education system are ongoing. Therefore, it is the right time to reflect upon the introduction and implementation of new methods describing the actual quality of educational processes.
The above-mentioned method of applying the EAV for evaluating the effectiveness of educational processes in public schools seems notable. This was acknowledged in several studies the paper made a reference to, and both in advantages and limitations outlined in the paper. Very much like other methods applied in the assessment of processes difficult to be quantitatively specified, it requires considerable caution in the formulation of conclusions. As far as the ongoing reform and determinants of Polish education are concerned, a greater focus upon this method of educational processes effectiveness assessment is required.