AP08857218 «Information technology for evaluating the scientific activities of universities, research institutes and their divisions»
Competition title: Ministry of Science and Higher Education of the Republic of Kazakhstan (MSHE RK), 2020–2022 (27 months)
Funding amount: 60.141 million tenge
Project abstract: In order to modernize the scientific, technological, and innovative development of the Republic of Kazakhstan, effective tools for evaluating the results of the research activities of scientists, universities, and research institutes must be developed. This makes it possible to verify whether the research process corresponds to the objectives defined at the planning stage and, if necessary, to adjust the course of these studies. The obtained assessments also serve as an effective instrument for planning the funding of such research. For the formation of research teams for joint studies, a typical approach involves selecting partners from among scientists who possess the relevant qualifications and experience in implementing similar projects. Establishing effective collaboration within scientific communities and the scientifically grounded formation of teams capable of successfully implementing projects are particularly important for project-oriented organizations, research institutes, and universities.
Funding amount: 60.141 million tenge
Project abstract: In order to modernize the scientific, technological, and innovative development of the Republic of Kazakhstan, effective tools for evaluating the results of the research activities of scientists, universities, and research institutes must be developed. This makes it possible to verify whether the research process corresponds to the objectives defined at the planning stage and, if necessary, to adjust the course of these studies. The obtained assessments also serve as an effective instrument for planning the funding of such research. For the formation of research teams for joint studies, a typical approach involves selecting partners from among scientists who possess the relevant qualifications and experience in implementing similar projects. Establishing effective collaboration within scientific communities and the scientifically grounded formation of teams capable of successfully implementing projects are particularly important for project-oriented organizations, research institutes, and universities.
Project Objective
Development of an information technology for the evaluation of the research activities of universities and research institutes of the Republic of Kazakhstan and their structural subdivisions through the development of citation indicators sensitive to instantaneous changes and the creation of an information system for assessing research performance. Integration into the developed information system of an intelligent partner search capability for conducting joint research.
Development of an information technology for the evaluation of the research activities of universities and research institutes of the Republic of Kazakhstan and their structural subdivisions through the development of citation indicators sensitive to instantaneous changes and the creation of an information system for assessing research performance. Integration into the developed information system of an intelligent partner search capability for conducting joint research.
Project Tasks
Development of an information technology for evaluating the research performance of scientists, departments, universities, and research institutes based on the calculation of various citation indices and integrated assessment indicators using intelligent data analysis methods, as well as the development of corresponding modules of an information and analytical system for assessing the results of scientific activities of universities and research institutes of the Republic of Kazakhstan. The system will provide automated collection and analysis of information from open data sources (Web of Science, Scopus, Google Scholar, etc.). In addition, it is planned to create a module for solving the task of selecting potential partners for collaboration within scientific and educational projects.
Development of an information technology for evaluating the research performance of scientists, departments, universities, and research institutes based on the calculation of various citation indices and integrated assessment indicators using intelligent data analysis methods, as well as the development of corresponding modules of an information and analytical system for assessing the results of scientific activities of universities and research institutes of the Republic of Kazakhstan. The system will provide automated collection and analysis of information from open data sources (Web of Science, Scopus, Google Scholar, etc.). In addition, it is planned to create a module for solving the task of selecting potential partners for collaboration within scientific and educational projects.
Project Implementation Stages
1. Analysis of existing scientific developments related to the evaluation of qualitative indicators of researchers’ scientific activities and the search for partners for joint grant activities. Identification of sources for collecting information on the research activities of scientists, universities, and research institutes.
2. Development of integrated criteria for evaluating the research performance of scientists, taking into account sensitivity to citations of publications by other authors. Development of an algorithm for ranking scientists, universities, research institutes, and their subdivisions based on the obtained integrated assessments. Development of a visualization module and interface within the software system for evaluating scientific activities.
3. Practical implementation of the information system for analyzing research activities. Development of modules for the automated collection of information on publications and their citations according to data from international scientometric databases (Web of Science, Scopus, Google Scholar) and other open data sources. Compilation of rankings of scientists, universities, and research institutes of Kazakhstan. Creation of a module for searching for research partners for the joint implementation of projects worldwide. Development of a user manual for the software system for the Ministry of Education and Science of Kazakhstan.
1. Analysis of existing scientific developments related to the evaluation of qualitative indicators of researchers’ scientific activities and the search for partners for joint grant activities. Identification of sources for collecting information on the research activities of scientists, universities, and research institutes.
2. Development of integrated criteria for evaluating the research performance of scientists, taking into account sensitivity to citations of publications by other authors. Development of an algorithm for ranking scientists, universities, research institutes, and their subdivisions based on the obtained integrated assessments. Development of a visualization module and interface within the software system for evaluating scientific activities.
3. Practical implementation of the information system for analyzing research activities. Development of modules for the automated collection of information on publications and their citations according to data from international scientometric databases (Web of Science, Scopus, Google Scholar) and other open data sources. Compilation of rankings of scientists, universities, and research institutes of Kazakhstan. Creation of a module for searching for research partners for the joint implementation of projects worldwide. Development of a user manual for the software system for the Ministry of Education and Science of Kazakhstan.
Expected Results
As a result of the research, methods for evaluating the results of the scientific research activities of scientists will be developed. A method for calculating the evaluation of the scientific research performance of scientists will be proposed which, unlike well-known indices (h-index, g-index, e-index, I-10 index), takes into account the entire body of citation information related to authors without losing information about their publications. The authors will also propose an integrated method for evaluating the performance results of scientists. A distinctive feature of this method is the construction of vectors whose components represent scalar evaluations of scientific activity results. These vectors are placed within a multidimensional metric space. The proposed methods are expected to be sufficiently flexible, as they will allow the adjustment of resulting evaluations through the selection of different formulas for calculating metric distances and coefficients that determine the citation relationships between scientists’ publications.
As a result of the research, methods for evaluating the results of the scientific research activities of scientists will be developed. A method for calculating the evaluation of the scientific research performance of scientists will be proposed which, unlike well-known indices (h-index, g-index, e-index, I-10 index), takes into account the entire body of citation information related to authors without losing information about their publications. The authors will also propose an integrated method for evaluating the performance results of scientists. A distinctive feature of this method is the construction of vectors whose components represent scalar evaluations of scientific activity results. These vectors are placed within a multidimensional metric space. The proposed methods are expected to be sufficiently flexible, as they will allow the adjustment of resulting evaluations through the selection of different formulas for calculating metric distances and coefficients that determine the citation relationships between scientists’ publications.
Project Team
Andrey Alexandrovich Beloshitsky, Doctor of Technical Sciences, Professor
Scientific Supervisor of the Project
Role in the project:
Project management, implementation of all project stages in accordance with the project schedule, and ensuring the achievement of the required results.
Scientific Supervisor of the Project
Role in the project:
Project management, implementation of all project stages in accordance with the project schedule, and ensuring the achievement of the required results.
Aleksandr Yurievich Kuchansky, Doctor of Technical Sciences, Associate Professor
Senior Researcher
Role in the project:
Development of methods and systems for evaluating the research activities of universities, research institutes, and researchers, taking into account the specific characteristics of the educational and scientific system of the Republic of Kazakhstan. Development of criteria and methods for selecting partners for joint scientific grant activities.
Senior Researcher
Role in the project:
Development of methods and systems for evaluating the research activities of universities, research institutes, and researchers, taking into account the specific characteristics of the educational and scientific system of the Republic of Kazakhstan. Development of criteria and methods for selecting partners for joint scientific grant activities.
Beibut Edilkhanovich Amirgaliyev, Candidate of Technical Sciences, Associate Professor
Senior Researcher
Role in the project:
Analysis of existing systems for evaluating the research activities of universities, research institutes, and researchers; development of mathematical models and methods; and development of algorithms for the university ranking system.
Senior Researcher
Role in the project:
Analysis of existing systems for evaluating the research activities of universities, research institutes, and researchers; development of mathematical models and methods; and development of algorithms for the university ranking system.
Aidos Agdarbekovich Mukhataev, Candidate of Pedagogical Sciences, Associate Professor
Researcher
Role in the project:
Identification of sources for collecting information on the research activities of scientists, universities, and research institutes; development of an algorithm for ranking scientists, universities, research institutes, and their subdivisions based on the obtained integrated assessments.
Researcher
Role in the project:
Identification of sources for collecting information on the research activities of scientists, universities, and research institutes; development of an algorithm for ranking scientists, universities, research institutes, and their subdivisions based on the obtained integrated assessments.
Sapar Nurakhmetovich Toksanov, Doctoral Student
Junior Researcher
Role in the project:
Development of modules for automated collection of information on publications and citations; development of a visualization module and interface within the software system for evaluating scientific activities.
Junior Researcher
Role in the project:
Development of modules for automated collection of information on publications and citations; development of a visualization module and interface within the software system for evaluating scientific activities.
Adil Ramazanovich Faizullin, Doctoral Student
Junior Researcher
Role in the project:
Development of integrated criteria for evaluating the research performance of scientists.
Junior Researcher
Role in the project:
Development of integrated criteria for evaluating the research performance of scientists.
Project Results
Interaction Scheme of Research Activity Entities
Parameters of Research Activity Entities
Citation Graph of Scientific Publications
Method for Determining the Distance Between Publications Based on the Semantic Similarity of Their Abstracts
Method for Clustering the Citation Graph of Scientists
Clusters of Scientific Publications
Merging of Scientific Publication Clusters
Thematic Model. Latent Semantic Analysis of Text
Method for Naming Clusters and Identifying Subject-Specific Scientific Spaces
Fuzzy Expert Survey Method for Naming Clusters
Method for Assessing Research Productivity
Researcher Identifiers and Evaluation Considering Self-Citations
Correlation Dependence Graph (Number of Publications vs. h-Index). r = 0.898
Correlation Dependence Graph (h-Index vs. Number of Co-Authors). r = 0.898
Evaluation of Collective Research Activity Entities
Selection of Research Activity Entities for Organizing Collaboration within Scientific and Educational Projects
Selection of Research Activity Entities for Collaboration in Scientific and Educational Projects
Information System for Evaluating the Research Activities of Universities, Research Institutes, and Their Subdivisions
ICT MONITORING
Progress and Effectiveness of Scientific and Scientific-Technical Projects and Programs
Progress and Effectiveness of Scientific and Scientific-Technical Projects and Programs
Successfully Completed on October 7, 2021
The project implementation is proceeding according to the calendar schedule, with an overachievement of the plan in terms of scientific publications.