brozura-meranie-EN

IMPACT MEASUREMENT
of Generation 3.0 Awarded Projects

GENERATION 3.0 CYCLE
System changes
We communicate with
stakeholders to integrate
the verified educational
approaches into
the education system.
Applying for
Generation 3.0
Evaluation
committee
The Lab
The aim of the two-day
bootcamp is to select the best
projects led by motivated teams
prepared to disseminate their
approaches to Slovak schools.
Awards
Measurement
We work with experts to
monitor the effectiveness
and dissemination
of awarded projects.
The best projects receive
a financial grant of € 15,000
and professional support
in monitoring their impact.
Personality Prepared for Changes
Natural Sciences and Technology Knowledge
EDUmap
Educational
approaches
online map.
Career and Entrepreneurial Skills
Social and Civic Competences
Innovations in the System of Education
EDUpoints
(Bratislava, Košice, Zvolen)
EDUpoint is a place
to educate teachers, school
principals, and parents
as well as to disseminate
educational approaches.
MentorExpres
In a three-month accelerator
programme, business experts advise
the best organisations in the Lab
on how to develop and disseminate
their approaches most effectively.
Give our children and young people
the chance to develop their full potential
and acquire the knowledge, values
and skills necessary for the 21 st century.
3

RESEARCH DESIGN
In the first year of the Generation 3.0 programme, we monitored
the effectiveness of awarded educational approaches. For each
of them, we had designed a research project with its objectives,
questions, and hypotheses which we were verifying during
the entire school year. The results are in the report you hold in your
hands. In it, we summarise the course of the research and show
to what extent the projects develop young people’s skills,
knowledge, and attitudes necessary for life in the 21 st century.
In this second year, we are modifying the ways of monitoring
the impact of awarded projects.
In order to meet our goal – to reach 15,000 teachers and learning
guides, it is not necessary for each project to carry out
time-consuming and costly research. After all, even the most
detailed analysis that confirms the impact of the educational
approach does not automatically ensure its integration into
the system. For that, the projects would have to be accepted
by all stakeholders and as widely as possible.
Through ongoing monitoring, we will be able to better advise
the organisations on how to develop their projects so that they can
be disseminated to schools as efficiently as possible. The evaluation
and result monitoring must be flexible and show that the approach
brings both changes in the education system and the potential
for dissemination.
We have modified the result evaluation strategy to gain answers
to three basic questions:
4

1. Is the educational approach effective?
In the first step, researchers will use professional literature
to find out whether such an educational approach already exists
and works somewhere in the world. They will assess whether
these conclusions suffice for the assessment of the effectiveness
of the educational approach under the Slovak conditions.
Consequently, they will prepare a research design and calculate
its approximate costs. They will also prepare a short study
with an estimate of the benefits and costs compared to similar
projects implemented in Slovakia.
2. Is the educational approach disseminatable?
The researchers will assess whether the approach is disseminatable
under the conditions of the Slovak education system
and whether the organisation has sufficient capacities to do so.
3. What are the lessons learnt for further dissemination,
evaluation and research activities?
We will use the feedback from people involved in the project
implementation and dissemination of similar educational
approaches.
5

C. S. LEWIS BILINGUAL HIGH SCHOOL
Personality Development Programme and the Transition of the C. S. Lewis Grammar
High School to an Inclusive Community
Education research expert: Martin Kuruc
The goal of the school is to produce graduates with a solid character and high-quality
education. That is why character education is a part of the school curriculum
as a whole. It is applied across all subjects and at all levels – including all, the pupils,
teachers, and the leadership. In the course of five years, the students are asked
in different ways about their motivation (how do I define success?), identity (who am
I?), mission (why was I born?), faith (what are my beliefs?), and the use of acquired skills
and attitudes in the future (what will I do with all these in the next 50 years?). Together
with their mentors – teachers, the students look for answers to these questions.
What We Measured
Hypothesis 1 (hereafter H): We assume that teachers show a significantly higher
proportion of positive than negative attitudes towards the inclusion
of disadvantaged pupils.
H2: We assume that in students of higher grades there is a significantly higher
proportion of positive attitudes towards inclusion of disadvantaged students
than in first-year students.
H3: We assume that the first-year students will show a smaller share of positive
attitudes towards inclusion of disadvantaged students at the beginning
of the school year than at the end of the school year.
Research question (focus groups): What are the elements of inclusion in social
relationships in the school?
Impact Measurement
There were 253 pupils and 19 out of 50 teachers participated in the research. Data
were obtained through a quantitative method using questionnaires and a qualitative
method through focus groups. In the case of H3 and focus groups, data were collected
at the beginning and at the end of the school year.
6

Research Results
H1: Following from the analysis of collected data, we can state that the H1 has
been confirmed. When there are 14 teachers in a school, all of whose attitudes
are pro-inclusive, and 5 whose attitudes are at least partially pro-inclusive,
it provides a good prerequisite for a gradual creation of pro-inclusive
atmosphere throughout the school.
H2: Following from the analysis of the collected data we can say that
H2 has not been confirmed. In the higher grades, there was not a significantly
higher proportion of positive attitudes towards inclusion of disadvantaged
students than in first-year students. A possible explanation may be that pupils
concentrate on mastering their quite demanding studies and have no internal
capacity to address these issues. It can also be related to the issue
of development. During this period, the students still need to cope with their own
changes. The adolescent children tend to close up so that they can find stability
within themselves and then open up to the world.
Still, based on the overall data, we can consider pupils’ attitudes
to be significantly pro-inclusive. 95% of students had a positive mindset
in all three components of attitudes – affective, behavioral, and cognitive.
H3: Following from the analysis of the collected data, we can claim that
H3 has not been confirmed, which means that the first-year students did not
show a smaller share of positive attitudes towards inclusion of disadvantaged
students at the beginning of the school year than at the end of the school year.
(To see the explanation of this result, please see H2.)
Focus groups: Elements of inclusion are manifested by the development
of informal relationships between students and teachers based on partnership
and helpfulness, which have been differentiated by mutual sympathy during
the school year. The informal atmosphere of the school, the opportunity to join
various activities and engage publically contribute to inclusion. However, the
amount of activities also brings the burden (sometimes, it seems to be excessive).
7

EDUMA
Online Live Library
Education research expert: Ivan Lukšík
The online live library is a video portal that uses storytelling to show children
and young people what it feels like to live with certain disadvantages and obstacles.
Storytelling is the best way of introducing any topic to children. There are dozens
of personal testimonies of “living books” at the onlinezivakniznica.sk website, which
can be used in teaching multiple subjects. Thanks to the encounters with the stories,
children can better empathise with other people and their situations, look
at things from another point of view, and learn to be empathetic. The online live library
strengthens students’ critical thinking and helps prevent prejudices against diversity.
What We Measured
Research Question 1: How have teachers who participated in EDUMA education
“Through Storytelling to Inclusion” changed their thinking and attitudes towards
inclusion? Has there been a shift towards critical thinking?
Research Question 2: How have the pupils with whom these teachers
subsequently worked using the “Through Storytelling to Inclusion” programme
changed their thinking? Has there been a shift towards critical thinking?
Impact Measurement
The research sample consisted of 12 teachers, 26 pupils in the experimental group
and 29 pupils in the control group. We used a questionnaire method. Teachers were
supposed to comment on the claims about Roma children, migrants, and so on before
and after the educational programme “Through Storytelling to Inclusion”. Then, they
used the programme as part of their teaching. Their pupils were tested before and after
the programme.
8

Research Results
Research Question 1: The results of the evaluation show that the teachers
who participated in the programme are able to work with their prejudices.
They take a critical look at their own prejudices, for doing which pro-inclusion
attitudes and critical thinking are essential. Teachers use inclusive elements
in the learning process and create conditions for the critical thinking of pupils.
Research Question 2: The results show that after completing the programme,
pupils take a critical look at their own ethnic stereotypes and start thinking
in a broader context. In relation to migrants, there has been a shift from rejection
and fear of diversity towards help in need, empathy, and consideration
of political contexts.
CIVIC ASSOCIATION OWL
ChemPlay - Advancement
Education research expert: Romana Kanovská
ChemPlay is a board game that introduces players to inorganic chemistry, its logic
and laws, and helps develop analytical thinking, critical thinking, and teamwork.
Its inventors are Ivana Kravárova and Adrián Hegedűš. During their studies
at a grammar school, they noticed that their classmates were not used to think
creatively. The students also did not know how to use theoretical knowledge
in practice even though chemistry provides loads of scope for creativity. The game
has become popular at schools and is available for sale to anyone who wants to learn
about the beauty of chemistry in a fun and effective way. Civic Association OWL is now
creating a mobile app as an advancement of the board game.
What We Measured
H1: We assume that pupils taught using the ChemPlay board game will show
a higher level of knowledge than the pupils taught frontally.
H2: We assume that students taught using the ChemPlay board game will show
a higher degree of internal motivation than the pupils taught frontally.
H3: We assume that pupils taught using the ChemPlay board game will show more
positive attitudes towards their teacher than pupils taught frontally.
9

Impact Measurement
There were 10 selected Slovak grammar schools (an experimental group of 5 schools
+ a control group of 5 schools) were involved in the research. The experimental group
consisted of 67 pupils, the control group had 65 pupils.
Data collection took place through a quantitative method in the form of questionnaires
at the beginning and at the end of the school year.
Research Results
H1: In the experimental group (schools using ChemPlay), the difference
in the level of knowledge between the entry and end results of the first year
pupils was moderately better (entry level of knowledge: 69.1%, end level: 79.5%).
In the control group (schools without ChemPlay), the difference in the level
of knowledge between entry and exit results of the first year pupils was
slightly better (entry level of knowledge: 69.4%, exit level: 76.9%)
It follows that the schools which used ChemPlay during the school year show
somewhat better results than the schools that did not use this board game.
Given that the board game has only been used rarely in schools, we can assume
that regular playing would lead to even more significant results.
H2 + H3: In pupils’ attitudes towards their teachers and their motivation to learn
chemistry, we did not record any significant differences between the control
and experimental group. This is probably due to the fact that the two selected
groups were very balanced and made up of very clever and motivated pupils.
Romana Kanovská
Director of the
National Institute
for Certified Educational
Measurements
Martin Kuruc
Researcher at the
Faculty of Education,
the Comenius University
Ivan Lukšík
Social and pedagogical
psychologist and lecturer
at the Faculty of Education,
the University of Trnava
10

WE WOULD LIKE TO THANK
The Generation 3.0 programme is funded from individual, corporate, and institutional
donors. Without listing their names upon their request, we would like to say a sincere
thank you to all individual philanthropists. By dedicating their time, know-how,
and finances, they contribute to the long-term sustainability and progress of the
programme.
Significant partners
Main partners Partners With support
11

generacia30.sk
nadaciapontis.sk