Avatud taotlusvooru "Nutikad lahendused noorsootöös" hindamiskomisjoni otsused

EEA and Norway Grants 2014-2021 Programme “Local Development and Poverty Reduction” Selection Committee meeting Date and time: November 26, 2021 from 11:00 to 13:30 (Estonian time) Venue: Cisco Meet web platform & MoSA, 7th floor, room Siil (Suur-Ameerika 1, Tallinn)  Open call for proposals „Smart Youth Work“ Selection Committee voting members:  Mr Marcus Ehasoo – chairperson of the Board of the Federation of the Estonian Student Unions;  Ms Krista Dreger – culture and youth work specialist at Harku Rural Municipality Government  Mr Juha Kiviniemi – planning officer at Verke, the Centre of Expertise for Digital Youth Work Finland. Observers:  Ms Gudrun Gudmundsdottir – Financial Mechanism Office, Senior Sector Officer, Health, Children and Youth, also representing the Financial Mechanism Committee;  Ms Katrin Olt – chief expert of the Department of Youth and Talent Policy, Ministry of Education and Research;  Ms Piret Marvet – representative of the donors, Royal Norwegian Embassy in Tallinn;  Ms Laura Pikkoja – representative of the National Focal Point; State Shared Service Centre;  Ms Kelly Poopuu – representative of the PO Implementing Agency, State Shared Service Centre;  Mr Ülar Vaadumäe – Ministry of Social Affairs, Smart Development Department, chief specialist, representing Programme Operator (PO). Minutes of the meeting Mr Ülar Vaadumäe gave an overview of the goals, targets and criteria of the call for proposals. Also, Mr Vaadumäe introduced the submitted projects and explained the tasks of the Selection Committee. For more, please see Annex 1 (PO presentation). Altogether 2 project applications were received within the deadline (April 15, 2021 at 17:00 o’clock local time) set for the call. As a result, the total grant amount applied for is 2,6 MEUR, exceeding the available grant amount by approx. 871 000 euros. The initial selection process was carried out in the summer of 2021 and the Selection Committee convened on August 3, 2021. The committee made its comments and proposals for improvement as well as suggestions on financing of projects. After that – and before any formal grant decisions were made – the Auditing Authority as well as the National Focal Point saw it necessary to carry out the selection process from the beginning in order to eliminate any accusation of conflict of interest. This derived from the fact that the 2nd tier of selection process, i.e. the Selection Committee had included experts from the Ministry of Education and Research (MoE) and one of the projects under discussion was Education and Youth Board that is a government institution subject to Minister of Education and Research. Therefore, the PO started a new selection process. Both projects were given an equal opportunity to improve their project proposals according to the deficiencies and comments brought out by the independent experts Ms Maarja Olesk and Ms Heleri Reinsalu. Programme Operator Implementing Agency (State Shared Service Centre) carried out the assessment of the project applications and were forwarded for evaluation by new impartial, independent and reliable experts. These experts were:  Evaluator 1: Ms Maarja Vollmer, sociologist, co-author of a research study "Using infotechnological means in youth work" (2016, Institute of Baltic Studies, e-Governance Academy)  Evaluator 2: Mr Anzori Barkalaja, head of the Education Innovation Centre of the University of Tartu in years 2015-2019, while also the international student start-up programme Digi Youth was launched  Evaluator 3: Mr Innar Liiv, PhD, Tallinn University of Technology , School of Information Technologies, Department of Software Science, Associate Professor  There were no project proposals receiving less than 50% of the total score, and therefore would have automatically received a negative decision (point 20.6).  As the evaluations of Evaluator 1 and Evaluator 2 were more than 30% apart for the application submitted by Education and Youth Board, Evaluator 3 was asked to evaluate this application as well. Ultimately, the evaluation scores of Evaluator 1 and Evaluator 3 were more closer to each other and according to the rules set (point 20.5), the average score of the two closest scores were used for the ranking of the projects, i.e. Evaluator 1 and Evaluator 3. Please see also Annex 2 (Ranking list of experts). Discussion Mr Vaadumäe opened the discussion with a remark that the impartial evaluators have concluded the ranking list as follows: 1. Education and Youth Board project application „Digital Leap in the Field of Youth“, in partnership with University of Tartu, Tallinn University, the Association of Estonian Open Youth Centres and the Estonian Association of Youth Workers = average score of 87,0 points and 2. CTF-Tech Ltd project application “Cyber Training Programme for Youth and Youth Workers“, in partnership with the University of Tartu and CybExer Technologies Ltd = average score of 77,5 points. Nevertheless, Education and Youth Board project proposal needed a 3rd opinion as evaluator No 2 Mr Anzori Barkalaja was very critical to the core content of the proposal and the evaluation scores of evaluators No 1 and No 2 were very different. Mr Juha Kiviniemi inquired about the results of the initial evaluation in comparison to the current one. Mr Ülar Vaadumäe showed the Selection Committee also the evaluation results of the previous independent experts and concluded that on the large scale the rankings and proportions between proposals have remained the same, but we have to take into account that in case of Education and Youth Board project there is also a very negative evaluation by evaluator No 2 that has been rejected in calculating the average score. Mr Ülar Vaadumäe summarised the evaluators’ comments – for both projects the profile and the capacity of the project applicants (incl. partners) is strong and reliable to carry out the proposed activities. CTF-Tech Ltd has focused on a rather narrow scope (raising cyber awareness among young people and youth workers) and lacks in outreach, so it cannot be hoped their intervention would bring about a wider improvement in raising the quality of youth work in local governments. Education and Youth Board has – on the contrary – taken up quite a wide and ambitious scope with different digital services envisaged to be developed (e-participation platform, non-formal learning outcome log, e-youth centre, e-games, etc.). Both applications lack in methods of ensuring quality engagement with young people with special educational needs, young people with a migrant background and young people in the risk of exclusion (vulnerable and marginalised young people). Mr Marcus Ehasoo opened by stating that the proposal submitted by CTF-Tech has been described well and the applicant is competent and has the capability to carry out the project. On the other hand, the proposal submitted by the Education and Youth Board is poor – scattered and unclear; it is questionable how this huge project would be managed in in the timeframe given (inadequate capacity). Mr Juha Kiviniemi agreed with Mr Ehasoo by explaining that the Education and Youth Board project tries to be “all over the place” with many topic focuses. Some activities, i.e. AI development for youth work is not viable yet for the next years to come, while in some activities it is not reasonable to create own new models as there are functional platforms already in place that can be adapted for youth work in Estonia. Mr Kiviniemi also expressed his regret that evaluator No 3 Mr Innar Liiv has not justified his scores and there is practically no information to go on in order to understand his reasoning behind the evaluation. Ms Krista Dreger brought out that from her point of view there is urgent need to raise the digital competence of youth workers to raise the quality of youth work, but the project proposal of Education and Youth Board does not reflect any consultation with experts in the field (who are the local municipalities to be included?) as well as the applicant has not included IT competence that is of essential importance to the successful implementation of the project. In order for new platforms to be successfully integrated into everyday working instruments there needs to be a mapping of digital tools and capabilities of local governments and their compatibility with Education and Youth Board systems. Ms Dreger hoped that the development of the e-platform would also boost the Youth Council activity. In regard to CTF-Tech project, Ms Dreger acknowledged that although the applicant is not a part of the traditional youth work scene, then their project substance falls into an important focus area dealing with uniting formal and informal education and raising the awareness in operating in the cyber world, its threats and raising the security of operations. -------------------------------- Mr Ülar Vaadumäe summarised that it seems that the Selection Committee members have indicated that in their opinion the proposal by CTF-Tech is to be considered better than the one by Education and Youth Board. That – in turn – falls in conflict with the ranking list of independent experts. Mr Vaadumäe brought out that according to the Selection Committee working procedures as well as point 20.7 of the call text, the Selection Committee may modify the ranking list of the project applications or propose only partial financing of projects in justified cases based on transparent criteria recorded in the minutes of the committee meeting. Mr Vaadumäe also admitted that although in case of Education and Youth Board project the evaluation score of evaluator No 3 was included into calculating the mathematical average, then the arguments for the scores are non-existent, while evaluator No 2 – whose evaluation score was rejected – has described many relevant deficiencies that the Selection Committee has subscribed to. Ms Gudrun Gudmundsdottir agreed that as a rule the Selection Committee possesses the power to modify the ranking list of projects formed by the average scores of independent experts. Ms Laura Pikkoja was hesitant how this change would be feasible, what would be arguments there and would it mean that the Selection Committee has to evaluate the projects similarly to the independent experts according to the evaluation criteria set. Mr Ülar Vaadumäe answered that in case of Education and Youth Board the Selection Committee members have brought out several deficiencies in the project that correlate with the comments from evaluator No 2, but what are not represented in the average score as the scores of evaluator 1 and evaluator 3 were included. Still, evaluator 3 has not justified his evaluation scores in any way, so basically there are arguments given only by evaluators 1 and 2. Mr Marcus Ehasoo stated that he sees the project by CTF-Tech to be a focused project with reachable goals and it should receive financing. On the other hand, the project by Education and Youth Board has taken up an excessively large scope that is unrealistic to be executed with the narrow time-frame and within the budget only 1,7 million euros. Additionally, as the reasoning for the budgeting remains unclear, it raises questions on the ability of the applicant to manage a project of such magnitude. Ms Krista Dreger added that the project by Education and Youth Board is unclear and does not include an IT partner, but in principle deals with essential concerns in youth work, especially important of them is raising the digital competence of youth workers. The CTF-Tech project is compact and deals with a very important topic regarding youth in the cyber world. Mr Juha Kiviniemi agreed that the project by Education and Youth Board is quite ambiguous and has it is unfortunate that the project has not included an IT partner. The project should re- focus prioritise its activities. Selection Committee decisions 1. The Selection Committee decided to modify the ranking list of independent experts and place the application submitted by CTF-Tech Ltd as No 1 and the application submitted by the Education and Youth Board as No 2. The reasoning for that can be seen above in the comments by the Selection Committee members in regard to the quality of the Education and Youth Board project as well as critical deficiencies brought out by evaluator No 2 Mr  Anzori Barkalaja in relation to the same project. Several shortcomings that were pointed out already by the previous Selection Committee had been left unchanged. Additionally, no arguments were presented by evaluator No 3 Mr Innar Liiv in his relatively high evaluation scores for the Education and Youth Board project. 2. The Selection Committee decided to award a project grant to the CTF-Tech project application „Cyber Training Programme for Youth and Youth Workers” in the total amount of 871 435 euros (decrease by 34,63 euros). 3. The Selection Committee decided to offer a project grant to Education and Youth Board project “Digital Leap in the field of Youth” in the total amount of 871 435 euros. As the cut in the grant applied for constitutes up to 50%, then the applicant can and should renew the project focus, while keeping the impetus on raising the capacity of youth work in local governments and discarding some activities of not as equal priority. The Selection Committee also stresses that the applicant should remove from the budget salaries to be paid from project resources for activities that are to be considered as operational responsibility of the occupation and should be reimbursed from the budget of the organisation. Before a formal grant decision is cast for the applicant, the renewed application should be positively evaluated in regard to financing. Mr Ülar Vaadumäe thanked everybody for their input and closed the meeting. Meeting chaired by (digitally signed) minutes taken by Marcus Ehasoo (digitally signed) Ülar Vaadumäe  EEA and Norway Grants 2014-2021 Programme: Local Development and Poverty Reduction Selection Committee meeting November 26, 2021 Open call for proposals „ Smart Youth Work“  Overview Project max duration: 30 months (starting from Dec 1 only 29 months left) Call amount: € 1,742,870 Project grant size: € 871,435 – 1,742,870 Project grant rate: 85%-100% Eligible applicants: all legal persons under public or private law, non-governmental organisation registered in Estonia. Open call is financed from the EEA Grants FM  Eligible activities:  developing smart youth work solutions. The innovative methodologies and information technology solutions developed must be based on the actual interests and needs of young people and be developed in cooperation with young people.  development work to increase the capacity of local governments and youth organizations and to implement the adoption of smart youth work solutions.  activities directed towards youth workers, which increase their capacity to use smart youth work solutions. Activities may include trainings, cooperation and networking meetings and seminars with partners, outreach activities for young people and other relevant stakeholders.  Targets (cumulative for the open call):  at least 50 specialists who work with young people and children trained to develop applications based on target group needs;  at least 48 local governments must participate in the project’s activities. Number Number Expected programme Indicator Baseline Baseline Target results value year value Share of local governments Outcome 1 Enhanced wellbeing of involved in programme 0 N/A 60% children and youth activities applying smart solutions in youth work Evaluation criteria of the open call Criterion Points % Need for the project and conformity with the goals of the 1 open call 25 27,17% 2 Substantive quality of intervention logic 25 27,17% 3 Strategic project management 20 21,74% 4 Profile and capacity of project promoters 10 10,87% 5 Project visibility and sustainability 10 10,87% 6 Cooperation with donor countries 2 2,17% TOTAL 92  Projects received within deadline:  Altogether 2 project proposals were submitted by deadline a) Education and Youth Board in partnership with University of Tartu, Tallinn University, the Association of Estonian Open Youth Centres and the Estonian Association of Youth Workers b) CTF-Tech Ltd in partnership with the University of Tartu and CybExer Technologies Ltd The total grant amount applied for is 2,6 MEUR, exceeding the available grant amount by approx 870 000 euros. Evaluation of independent experts  Evaluator 1: Ms Maarja Vollmer, sociologist, co-author of a research study "Using infotechnological means in youth work" (2016, Institute of Baltic Studies, e-Governance Academy)  Evaluator 2: Mr Anzori Barkalaja, head of the Education Innovation Centre of the University of Tartu in years 2015-2019, while also the international student start-up programme Digi Youth was launched  Evaluator 3: Mr Innar Liiv, PhD, Tallinn University of Technology , School of Information Technologies, Department of Software Science, Associate Professor  There were no project proposals receiving less than 50% of the total score, and therefore would have received automatically a negative decision (point 20.6).  Ranking list of experts Education and CTF-Tech Ltd Youth Board Selection criteria No Criterion Max score Average score Average score Need for the project and conformity 1 with the goals of the open call 25 25,0 20,0 Substantive quality of intervention 2 logic 25 22,5 19,0 3 Strategic project management 20 19,5 19,0 Profile and capacity of project 4 promoters 10 10,0 8,0 5 Project visibility and sustainability 10 10,0 9,5 6 Cooperation with donor countries 2 0,0 2,0 TOTAL 92 87,0 77,5 Selection Committee members Voting members Mr Marcus Ehasoo– Chairperson of the Board of the Federation of the Estonian Student Unions; Ms Krista Dreger – Culture and youth work specialist at Harku Rural Municipality Government; Mr Juha Kiviniemi – Planning officer at Verke, the Centre of Expertise for Digital Youth Work Finland Observers The representatives of the National Focal Point and Programme Operator Implementing Agency (both State Shared Service Centre) as well as the donor states (Norwegian Ministry of Foreign Affairs and Financial Mechanism Committee) shall be invited to participate as observers. Selection Committee tasks The goal to confirm the list of project applications suggested to be financed and the list of project applications not to be financed and forward its decision to the Programme Operator Implementing Agency (i.e. State Shared Service Centre) for formal decision making. The tasks  review the ranking list of project applications resulting from the mathematical average of individual expert evaluation scores;  confirm the list of project applications suggested to be financed and the list of project applications not to be financed;  modify the ranking list of the project applications or propose only partial financing of projects in justified cases based on transparent criteria recorded in the minutes of the committee meeting. After the Selection Committee  Chairman of the Selection Committee and the PO will sign the minutes of the meeting, where decisions are laid down in a transparent and accountable manner describing which project proposals shall be awarded a grant and which proposals will be rejected.  PO Implementing Agency will cast a decision to award a grant or decline awarding a grant based on the written recommendation of the Selection Committee. Discussion and decisions  Aitäh! Takk! Ülar Vaadumäe ([email protected]) Martin Karro Riigi Tugiteenuste Keskus [email protected] Meie 02.12.2021 nr 2-6.3/3917-1 Avatud taotlusvooru "Nutikad lahendused noorsootöös" hindamiskomisjoni otsused Lugupeetud Martin Karro Palume Riigi Tugiteenuste Keskusel kui Euroopa Majanduspiirkonna ja Norra toetuste 2014– 2021 programmi „Kohalik areng ja vaesuse vähendamine“ rakendusüksusel langetada toetuse rahuldamise otsused avatud taotlusvooru „Nutikad lahendused noorsootöös“ raames laekunud projektitaotluste suhtes, mille on kinnitanud 26. novembril 2021. aastal kohtunud taotlusvooru hindamiskomisjon. Palun rakendusüksusel viia läbi riigiabi analüüs rahastamiseks soovitatud projektide suhtes. Samuti palun edastada Haridus- ja Noorteametile hindamiskomisjoni seatud tingimused rahastuse pakkumiseks, mis eeldavad taotlejalt projekti mahu ja eelarve olulist vähendamist ning uuendatud taotlusele täiendava hinnangu andmist (lisatud). Lugupidamisega (allkirjastatud digitaalselt) Kristiina Hunt nutika arengu toetamise osakonna juhataja Lisa: Minutes of the SC meeting (November 26 2021)_FINAL Lisaadressaat: Haridus- ja Teadusministeerium Ülar Vaadumäe 626 9249 [email protected] Suur-Ameerika 1 / 10122 Tallinn / 626 9301 / [email protected] / www.sm.ee / registrikood 70001952