Quadrennial Evaluation 2017-2020: Announced disaster of Brazilian graduate studies.

Abstract

The suspension of the current CAPES four-year evaluation process, for the period from 2017 to 2020, planned to be carried out this year, is quite worrying and quite unusual. This process, in the final stage of implementation, now suspended by a court decision on the initiative of the Federal Public Ministry, even if as a preliminary injunction, directly affects 4,650 Stricto Sensu Graduate Programs, 544 Institutions, involving about 105,000 Professors and 293 thousand students spread across the country.

Inconsequential, the measure can bring immeasurable damage to the National Postgraduate System (SNPG) as not only the country's academic community is affected, but also the entire country, which represents the results of the CAPES evaluation process. within the scope of the National Science and Technology System (SNCT). These results not only represent fundamental inputs for measuring advances in science and technology in the country, but also indicate the dynamics of growth of highly qualified human capital that is delivered to society. In addition, they interfere with the normality of academic activity, as they impact the life of the researcher, whether teacher or student, and above all institutions regarding the external perception of the quality of their Stricto Sensu Graduate Programs and the related investments undertaken or planned.

Our country, already so lacking in investments in Science and Technology, with immense asymmetries in terms of opportunities for training highly qualified personnel (masters and doctors), with repressed demands regarding its social and economic development, becomes hostage to measure like this, which causes a setback and, above all, impacts the credibility of CAPES as a heritage of the Brazilian scientific community. Interrupting the evaluation will certainly contribute to aggravating the country's situation in terms of the gap still observed between its volume of scientific production, in the 14th position in the world, and the qualification of this production in international terms measured in Citation by Publication (CPP) which refers us to to 77th position.

Why is an evaluation process interrupted after the entire four-year reference cycle has elapsed? The criteria were already known by the entire academic community and had been built since the beginning of the cycle in question. It should be noted that, above all, over the last 2 years it has been carried out in a very participatory way and with total transparency.

On the other hand, it must be considered that changes always bother those whose interests are considered compromised, even if the resulting impacts represent advances of a comprehensive and systemic scope, as is the case of the CAPES evaluation process. The current four-year evaluation process of the National Postgraduate System represents a major advance in its various dimensions. Characterized as a consolidated participatory process - the postgraduate academic community is represented at CAPES by 49 Knowledge Area Coordinations, each one formally composed of 3 Researchers chosen from among their peers -, historically the criteria have been widely discussed and improved to each evaluation cycle performed. The on-screen evaluation cycle was no different. Quite the contrary, since especially since 2019 they have been structurally improved. Preventing this improvement on the grounds that the principles of "predictability" and "non-illicit feedback" must be respected may be arguments that hide interests based on attempts to maintain the status quo in a comfort zone that does nothing to improve the quality of life. SNPG quality. Who cares about all this? I do not believe, due to the boundary conditions, that those who defend a science and technology of quality, towards the use of international standards, even if it is understood that these are still targets to be achieved. After all, continuous improvements need to be undertaken, even if gradually, respecting the rhythm and characteristics of the areas of knowledge. All areas of knowledge are important and diversity and their own characteristics need and were being respected. On the other hand, we understand that all areas, without distinction, must have the common purpose of reaching the highest levels of academic excellence of international standard that the country needs to achieve.

However, in view of the young evaluation process and CAPES's degree of maturity, it is observed that these purposes could only be achieved through a process built gradually over the four-year cycle itself. Is it ideal? Or would it be better if all criteria and indicators were defined before the evaluation cycle? Thus, in view of the complexity of the process, there has been a consensual understanding, within the scope of the group of actors of the SNPG, in a historical procedure, that the construction of the criteria must be carried out throughout the cycle because it is a comparative process, whose elements are not available at the beginning of the cycle. This was the procedure that proved to be viable for the current quadrennial. We believe that this was the best procedure to be used and as a transitional regime for a model of prior definition of indicators and criteria for the next four-year period.

The current four-year evaluation process, therefore, was built with this purpose in mind and is characterized by enormous advances in terms of its criteria and quality indicators for the intellectual production of Graduate Programs. Among other advances, the valuation of the social and economic impact of the academic activities of the PPGs and the new metrics for the evaluation of scientific publications, through the process called Qualis Referência (QR), which seeks to standardize the evaluation criteria of publications between areas of knowledge. This QR process performs the stratification of the set of scientific publications, respects the specific characteristics of the areas and uses, in an unprecedented way, indicators based on international evaluation metrics. For the first time, too, social impact measures would be applied.

If the current evaluation process is suspended, the damage will be enormous, as the same criteria used in the previous four-year period cannot be applied automatically. The data collection system of the Graduate Programs and the entire computational environment necessary for the evaluation, from its preliminary stages to its effective realization, have been intensively parameterized for at least two years. The resumption of the process, based on the parameters of the last four years, would only interest those who were not willing to face the challenges of seeking to improve the quality of the PPGS and, consequently, were always dissatisfied with the advances in the evaluation process that had been achieved. since the end of the last four-year period.

Without going into the merits regarding the actions, or the lack of them, at CAPES, it is time to defend this funding agency and respect its evaluation system.