NPA 2019-10 (A), (B),(C)

European Union Aviation Safety Agency Notice of Proposed Amendment 2019-10(C) Measurement of the safety key performance indicator and safety performance indicators in the SES Performance and Charging Scheme Material for the implementation and measurement of the safety key performance indicator (SKPI) and safety performance indicators (SPIs) for the Third Reference Period (RP3) of the SES Performance and Charging Scheme (Commission Implementing Regulation (EU) 2019/317) RMT.0723 EXECUTIVE SUMMARY The objective of this Notice of Proposed Amendment (NPA) is to provide technical material regarding the implementation and measurement of the SKPI at the level of air navigation service providers (ANSPs) and the SPIs at both the State and ANSP level. The NPA and the indicators referred to above are linked to Commission Implementing Regulation (EU) 2019/317 laying down a performance and charging scheme in the single European sky and repealing Implementing Regulations (EU) Nos 390/2013 and 391/2013, being the Third Reference Period (RP3) of the SES Performance and Charging Scheme. This NPA proposes AMC and GM appropriate to the requirements of the SKPI of RP3 of the SES ATM Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) 2019/317. The proposed amendments are expected to decrease the safety-reporting burden and reduce regulatory burden when compared with the AMC and GM for RP2 of the SES Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) No 390/2013. Further, the amendments are expected to facilitate stakeholders in complying with the safety performance requirements of the above-mentioned Commission Implementing Regulation. The SKPI reporting is restricted to ANSPs and, wherever possible, the SPIs will be calculated using occurrence data that has been reported to the European Central Repository under Regulation (EU) No 376/2014. Once consulted upon, the final material will be provided to the European Commission, whereupon it will be incorporated into a Commission Notice in relation to the SES Performance and Charging Scheme Regulation. As such, it will not be published as AMC and GM through an EASA ED Decision nor as an Opinion. Please note that the material is presented in three sub-NPAs: — NPA 2019-10(A): the Explanatory Note; — NPA 2019-10(B): the Annex to the Explanatory Note, containing the material that is proposed to be adopted into a Commission Notice, which further describes the SKPI and SPIs, as defined in Commission Implementing Regulation (EU) 2019/317; — NPA 2019-10(C): the Appendix to the Annex, providing the questionnaire and associated verification guidance for the Effectiveness of Safety Management (EoSM) SKPI. This will also be adopted into the Commission Notice. Action area: Safety; systemic enablers; safety management Related rules: Commission Implementing Regulation (EU) 2019/317 Affected stakeholders: ANSPs; Member States (MSs) Driver: Efficiency/proportionality Rulemaking group: Yes Impact assessment: None Rulemaking Procedure: Standard 1.5.2018 19.9.2019 2019/Q4 2019/Q4 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 1 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Appendix to SKPI — Verification of the ANSP EoSM by the NSA/competent authority Component 1: Safety Culture Study Area 1: Development of a Positive and Proactive Organisational Culture Level A Level B Level C Level D Question 1.1 Informal Defined Managed Assured Arrangements An intelligent and effective organisational Concept of Safety: Concept of Safety: Concept of Safety: Concept of Safety: culture is one which is responsive to the hierarchical differences in an Employees believe that safety goals will be Employees contribute to safety by highlighting The organisation recognises that safe Management systems recognise that change organisation. Differing functions and achieved by complying with rules and deficiencies in rules and procedures. provision of services is something it can can cause instability within the organisation. It roles in an organisation have different regulations. achieve through the expertise and experience actively engages and prepares to manage this The organisation is developing processes to of its staff, not simply by defining rules and instability, including the need to prepare views of risk, different risk disposition. People, especially front line staff, are support employees’ ability to share safety procedures. people for changes that may affect safety. Therefore, they have different values and considered the principle cause of accidents lessons learned with other teams or groups. views about safety. and incidents. Behaviour based safety interventions and campaigns are recognised as having Sanctions are applied by management when limitations and alternative strategies non-compliances are found. explored. Safety Culture Safety Culture Safety Culture Safety culture Safety culture is informal and applied only in Safety culture is applied in both operational The organisation acknowledges the need to Safety is understood to be the responsibility of the operational parts of the organization. and support functions. consider safety culture and organisational the organisation as a whole. The organisation culture together, but maintains the two as includes the contribution to safety by non- separate concepts. operational areas in its safety planning. The value of safety in the organisation is Organisational culture and safety culture are recognised and promoted through considered and managed as the same thing. engagement and consultation with staff. This engagement and consultation values diverse views of safety and respects difficult and challenging questions TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 2 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Safety Interventions and enablers Safety Interventions and enablers Safety Interventions and enablers Safety Interventions and enablers Rules and procedures are adapted based on In addition to adapting rules and procedures The organisation’s risk landscape is built from The organisation actively seeks diverse views lessons learned from occurrences. following safety occurrences, the organisation multiple perspectives – that of employees in of safety as a means to drive safety analyses its risks more strategically. different roles in the organisation and interventions. especially involving front-line staff. Processes are in place to ensure that any organisational member knows that a safety concern will be escalated following the issue being raised, explored and consensus reached on the need for action. Safety resources are used in a flexible manner that is targeted and safety activities are resourced and managed within business planning and reporting processes. The SMS The SMS The SMS The SMS Only applied on an ad-hoc basis There is an awareness that the SMS is a tool to The limitations of the SMS are acknowledged The Safety Management System (SMS) be used and the need for training some roles and organisations embark on both training encourages employees to question No SMS for non -certified organisations in safety is identified and begun. that supports intelligent use of the SMS (not procedures, practices and people to improve just applying it) and the evolution of the SMS safety performance. Safety is managed and in ad-hoc manner through progressive change. The organisation trains employees to fulfil their safety management responsibilities through developing the capability of those engaged in safety and managers who have an accountability for safety. Guidance — all levels Organisational culture embraces ‘safety culture’. Organisational culture includes the organisation as a whole and embraces the way that business decisions cascade through an organisation as well as the existence of subcultures which have their own perspective of safety, values and ‘tribal knowledge’, for instance the ATCO and engineering communities. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 3 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Differing functions and roles in an organisation have different views of risk, different risk appetites and, therefore, different perspectives of safety — which is in keeping with the perspective that the organisational culture brings. As a result, differing roles and functions see safety differently with respect to the way that they build safety into their work. The approach to these different values and views of safety, how they are recognised, reconciled and translated into actions provides an indication of the management’s approach and commitment to safety. An intelligent and effective organisational culture will embrace diversity, using the perspectives that such views bring to build a richer and deeper understanding of how the organisation performs and delivers safe provision of services. The choices made in managing the business, including safety and safe production, involves trade-offs, the consequences of which influence an organisation’s culture. Understanding the decision-making of managers who have both the accountability and the authority to deliver or facilitate the delivery of solutions to safety concerns is one way to explore organisational culture. This includes business decisions about the allocation of resources and budgets in an organisation. These trade-offs reflect policy and business choices made by the ANSP as well as those that are externally driven. For example, the business strategies that ANSPs adopt to meet the requirements for the SES RP3 targets in all Key Performance Areas (KPAs). Guidance for the Defined Level Concept of Safety: The key difference between Level A ‘Informal Arrangements’ and Level B ‘Defined’ is that whilst the organisation still ‘enforces’ safety through adherence to rules and procedures, there is a growing realisation that this approach has limitations. This may be because there are repeated behaviours by people that the organisation attempts to control with very limited effect. In practice, this means that the reliance and underlying belief that only rules and procedures ensure safety is fundamental as is the confidence in behaviour-based safety. Rules and procedures cannot be expected to cover all possible operational situations and eventualities. They are underspecified — they cannot cover all possible situations. As a consequence of this, continuing to add procedures and rules can make an operation less safe. This notion of safety will be beginning to be understood by ANSPs at the ‘Defined’ level, but not acted upon. Critical to facilitating this understanding is the way that safety departments undertake the investigation of reported occurrences. Arrangements need to be in place that recognise these ways of thinking about safety. Safety Culture Organisational decisions around resources and efficiency lead to consequences which are all perceived as degrading safety such as:  insufficient operational resources to manage demand requiring the imposition of ATFCM measures (leading to delay performance worsening);  changes of watch rosters to adapt capacity to demand that are beyond agreed rostering guidelines;  an increase in additional attendances (overtime);  insufficient slack to enable secondary operational duties to be undertaken;  engineering service level agreements slipping; TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 4 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI  training for new projects slipping;  fatigue is perceived as increasing. These issues reinforce the reality that decisions that are made in the non-operational sections of an organisation influence the safe provision of services. An ANSP at the Defined Level begins to listen to the others’ views but will have a reluctance to act upon what is heard. Therefore, concerns may be raised, but are rarely if ever pursued by those who the discussions take place with. As a result, the views of risk of those managing and those being managed grow ever greater apart. This will, therefore, shape the safety culture as well as organisational in-house employee surveys. Indications that an organisation is at the ‘Defined’ level may also include:  safety culture is acknowledged as necessary and is implemented as per the relevant regulatory guidelines and management system requirements;  there are no discussions around quality service delivery versus safe provision of services, because it is the perception that there is no point. It is not the place for employees to challenge management. Signs that the organisation is not yet at the ‘Managed’ level may include:  senior management presence in operational spaces at times when there are delays, but never during safety events;  human error and deviation from rules and procedures is still the principal focus of safety investigations and interventions;  management does not involve the operational community in identifying ways of improving safety performance;  people may be stigmatised for repeatedly raising safety concerns that they have;  training is used as a corrective and disciplinary action;  investigations micro-match procedures to what happened in normative ways. Safety Interventions and Enablers In addition to adapting rules and procedures following safety occurrences, the ANSP analyses its risks more strategically. Safety interventions, or safety mitigating actions, do not include systemic or structural solutions — instead, they just consider human actions or technical failures associated with specific occurrences. The SMS ANSPs at this level can be expected to have implemented a Safety Management System (SMS). In the process, the way that risk is measured and understood by those involved with safety, and the nature of safety improvements and interventions has become gradually more nuanced and progressive as a result as the implementation. As a result, the internal safety discourse begins to change with the ensuing safety measurement, and safety promotion is more visible. The safety discourse will be characterised by a lack of transparency — safety teams and managers, for example, leading the discourse with little structured or formal inclusivity of others in the organisation. Safety improvements are limited to what guidance is given by the SMS. There is an overwhelming confidence that following the SMS safety will be delivered. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 5 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Managed Level Concept of Safety: At the ‘Managed’ level, there is recognition that staff contribute to the safe provision of services through the way that the operational tasks are undertaken, including the way that trade-offs in the operation and beyond are taken. These rely on an intelligent use of strategies that are sensitive to operational risk and that achieve safe provision of services. For example, if ATFCM measures are needed, then it is known that if the need is there to do so, then there will be no criticism around the consequences on service provision, but there may be enquiries to gain a broader appreciation of the context. The impact of this evolutionary shift is that there will be a gradual change to less use of disciplinary and behaviour means to sustain safety and, where deemed appropriate, the change to identifying structural factors in the operational environment that shapes safety events and making changes there. There is a recognition that ‘people create safety’ in ways that cannot be encapsulated in rules and procedures alone. More specific activities might include:  the use of traffic management techniques to allow those involved in an event and who have to file an occurrence report to have the time to do this, and to recover from the event;  where there are competing demands made for limited operational resources, then safety is an explicit facet of decision-making where appropriate — the safe provision of services will be embedded in the trade-offs;  managers and supervisors develop a view of how the safe provision of services is by engagement with operational staff — leading to an informed discourse that develops confidence in organisational decision-making;  managers and supervisors actively seek the views of both the operational and non-operational community to gain an informed view about organisational safety, which may lead to a better understanding of how effective safe production can be enhanced;  managers and supervisors make themselves available when staff wish to discuss safety concerns with them;  staff representative organisations meet regularly to discuss and engage about safety;  organisations accept that procedures and rules cannot fully describe every eventuality. As such, they do not rely on new or additional rules and procedures as the only safety intervention, because they know that this can introduce new risks and without addressing structural issues. Safety Culture Indications that an ANSP or organisation has reached Level C ‘Managed’ can be found in the way that the ANSP has transitioned from the organisation seeing safety culture as a distinct independent entity, towards viewing it as a part of the overall organisational culture. The emphasis in this change can be seen in that the ANSP engages with those who work within it. At Level C, the value or benefit of safety in an organisation is recognised and promoted by managers and supervisors. Important indications are:  the use of organisational resources to develop safety education;  whether safety is integrated into business planning, including provisions for safety in the long-term investment planning; TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 6 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI  the safe provision of services versus quality of services is discussed. Safety Interventions and Enablers ANSPs at the ‘Managed’ level have evolved processes and mechanisms that use means other than occurrence reporting to assess, understand and manage risks. These processes and mechanisms have evolved beyond relying solely upon the use of the attribution of casual-factor taxonomies from occurrence reporting alone as it is recognised that this alone is limited and provides an incomplete risk landscape because:  it may not reflect the actual frequency of such events; the processes and mechanisms need evolving to encourage people to report because there is little seen to happen once a report has been filed;  there might be inconsistencies between incident investigators that lead to a lack of confidence in the casual-factor attribution;  safety interventions derived solely from causal attribution are seen to yield limited effectiveness or not to be able to find suitable solutions. As a result, complementary techniques are identified, examined, and experimented with and begin to be used in occurrence reporting, although incrementally at first. Techniques that may be used but are not limited to:  the inclusion of human factors investigation narratives;  the use of ‘second stories’ to gain an understanding of not just ‘what’ happened but ‘how’ the event occurred;  exploring ‘why did it make sense to them’;  the scope of the occurrence and incident investigation is broad and encompasses a larger sample of accounts including those outside the ANSP;  the use of aircraft operator narratives and flight data;  an explicit recognition that the operational context is complex and, therefore, what and how happened can be better understood by exploring the interactions between actors and system components as well as the multiple views that are used to produce a composite view of the event;  using the understanding of the operation that comes from observing safe production in practice to develop an understanding of typical ATC operations;  the introduction of investigator competence training and inter-investigator consistency schemes along with continuing professional development to enhance investigation skills;  expanding the risk landscape by taking the views of the wider organisational and explore path dependency (history as cause). The SMS An ANSP at the ‘Managed’ level will have worked with an SMS for some years, having implemented it and updated the safety management manual that underpins the SMS in part by a process of continual improvement, in part because of the need to comply with regulatory requirements, and in part because of experience with the development of the ATM systems. A move from strictly following the SMS to intelligent application of the processes can be seen. This may be driven as a result of the experience with applying the SMS in a range of changes within the ANSP, for instance across a range of technical systems with increased complexity. Additionally, there will have been TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 7 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI new stakeholders, e.g. engineering teams, change management, business risk, supply chain and software engineering that will contribute to different issues and perspectives. The SMS can consume resources as it is applied uniformly across all projects or within an ANSP’s activities. ANSPs can elect to develop proportionate applications of the requirements of an SMS. In so doing, progressive and intelligent application of the SMS provides evidence of an ANSP or organisation that is functioning at the ‘Managed’ level. Guidance for the Assured Level At Level D, ‘Assured’, safety should be considered as a property that is created within the organisation, not something that the organisation has. Safety is viewed as the domain of the organisation as a whole, not simply a component of operational departments and a selection of non -operational departments. The ANSP recognises that non-operational elements of an organisation contribute to the safe provision of services. Concept of Safety: Change brings with it numerous challenges and threats to sustaining performance, as well as to managing resources across the organisation as a whole. Such threats and challenges are necessarily organisation-wide and will involve third parties and many other actors. At the ‘Assured’ level, the ANSP’s SMS is designed around the recognition of the influence, effects and consequences of change on the safe provision of services, including how they affect people. It will make provision for this in business and safety management systems, including assessments and mitigations of change as both business and safety risks. More specific characteristics include the following:  the ANSP is sensitive to the balance between design changes at a late stage and its impact on implementation, including training and user confidence;  accountable managers who have to accept the change draw from the widest group of actors and work with them to determine a perspective of how the change is being implemented, as well as the preparations for training and readiness for the change;  processes are used that assess the quality of transition training at all stages of its design and implementation and changes that flow from changes in the design;  the management of change processes extends beyond the actual implementation date and includes post-implementation activities, including formal and informal verification of the design, the way that work has changed, review of performance, and adequacy of training;  sustaining an operational service throughout the transition steps will demonstrate preparedness to limit the scale of the operational task until it is agreed to increase the scale of the operational task beyond any restricted levels of service delivery. Safety Culture At this level, the ANSP recognise and implement safety as part of the overall organisational culture. In practice there are inevitable trade-offs between production–efficiency–safety–business planning. The ANSP will have evidence of formal and informal processes that accord an appropriate priority to safety. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 8 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI It is in the decisions that are made that balance and reconcile these conflicting demands, and reconcile the resource implications, that the value of safety can be seen. For example:  situations that are assessed to influence safety are seen as opportunities to develop a stronger and more effective safe service delivery process;  the need to pursue a strategy that is perceived as threatening safety by the operational community is managed in ways that are transparent and open to challenge;  it recognises the need to gather the knowledge behind fears, concerns and perceptions, and to engage meaningfully with the worldview that this brings. Safety Interventions and Enablers Organisational approaches to learning lessons recognise that there are limitations to classic and current approaches to safety processes. An organisation that is sensitive to this recognises that there is a learning potential in examining the formal processes and system of lessons learned, that is, the SMS at each and every step of the life cycle of an occurrence report. For example:  initial filing of the occurrence report;  the way that the reporter and others involved in the event were managed and cared for;  the process of managing those people at the time of the event, i.e. release from an operational position;  the quality and value of the initial occurrence report;  formal investigation processes and systems;  recommendation generation;  feedback loops;  safety oversight and review committees;  safety data propagation. To support learning from safety occurrences, investigators should be provided with dedicated continuous professional development to enhance both their understanding of safety and their investigation techniques. Investigators should be aware of the models of accident causation that they are using. The SMS The SMS will encourage challenge and critique as part of its contribution to a safer and more effective ANSP. Challenge and constructive critique are means of a feedback loop that can provide fundamental information about how the work system is behaving and ways to make structural changes. The SMS will emphasise the limitations of safety mechanisms and provide a clear evaluation of the strengths and weaknesses of the orthodox safety interventions. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 9 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 1: Safety Culture Study Area 1: Development of a Positive and Proactive Organisational Culture Level A Level B Level C Level D Question 1.2 Informal Defined Managed Assured Arrangements A just and open climate for reporting and Just Culture Just Culture Just Culture Just Culture investigating occurrences Management does not see the need for any Management and employees recognise the The organisation has established policies and Just culture has evolved through several Note A thorough reporting and activity or dialogue with the staff in this area. need to have Just Culture, in order to procedures to support Just Culture principles. iterations and development of Just Culture investigation process should begin with encourage reporting. They enter into After initial training and education across the policy, principles, processes and philosophy. notification, data gathering, dialogue internally, including the union and organisation, continuation training and The organisation has learnt how to measure reconstruction, analysis, safety the staff association. education is provided. the acceptance of Just Culture principles and recommendation and implementation of recognises the limitations of such remedial actions, resulting in final The fundamental attribution error is common Where decisions around ‘acceptable’ and measurement. reporting, exchange of lessons learned in occurrence investigation. The difference unacceptable behaviour are made, a process and effective monitoring. between acceptable and unacceptable is in place that arbitrates such decisions with There is evidence that the application of Just behaviour is misunderstood and misapplied. representatives, trained for the task. Culture is unaffected by changes in the organisation. Staff conditionally support Just Culture principles and management’s commitment Lessons from within the organisation and towards it. across different industry sectors are used to enhance the organisation’s approach to Just Culture. The value of reporting The value of reporting The value of reporting The value of reporting The value to individuals of reporting There is recognition that reporting The value of reporting is recognised, but the Reporting is seen as one source of safety occurrences is seen as low, because the risk occurrences has the potential to contribute emphasis in undertaking Just Culture is on the intelligence that contributes to a better of consequences is high. to safety. However, this is limited to consequences for individual actions in the understanding of how the operation and There is a perception that there is no circumstances where: most part. organisation function. contribution to safety by filing an occurrence.  an occurrence will not bring criticism or Reporting and investigation principles and The focus of occurrence reporting is around consequences upon the reporter, or processes are predominantly human-centric how safe and effective system performance  an occurrence is perceived as of serious in their attribution of causes of occurrences. can be sustained and enhanced. safety concern (to the reporter) or to the aggrieved party. All levels of the organisation are aware and The focus of reporting and investigation is on accept the difference between acceptable safe service provision, not as a mechanism for The application of just culture is viewed by and unacceptable behaviours. social control. The emphasis is on safety and individuals as inconsistent and unreliable. not the consequences of unsafe events TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 10 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI The Reporting culture The Reporting culture The Reporting culture The Reporting culture The reporting culture is one where there is an The reporting culture is one of ‘blame and awareness of the need and benefits for An open reporting culture is present where Just culture is seen as in the service of safe shame’. reporting but that the trust in the reports are filed. However, the value of service provision. As a result, open reporting organisation and processes are lacking. reporting is devalued by the limitations of the is perceived by staff as a means to Many events go unreported. reporting and investigation processes contributing to safe production and shaping Reporting and investigation processes, across themselves e.g. feedback to reporters, quality their future operational environment. the organisation are in the formative state of of recommendations and recommendation building a Just Culture. Reporting events is tracking (which is not incorporated into ANSP A competency scheme for investigators is common, but many are unreported. Business Management processes). applied. Post investigation quality questionnaires are used routinely. Disclosure Disclosure Disclosure Disclosure Disclosure of occurrences to external bodies Disclosure of occurrences to external bodies is Within legal limits, the organisation's safety The ANSP follows a clear and published policy are made with little if any protections for the identified as a business risk as well as a data are sufficiently protected from external that addresses the interface with the judiciary reporters of occurrences. deterrent to open reporting. interference (e.g. press, television, social on Just Culture matters. media). Internally, occurrence data is shared The organisation does not support those who The ANSP embarks on a dialogue with widely and anonymously. report and are subject to judicial action as a competent authority and the judiciary to result of disclosure to external agencies. develop principles and protections for Internally, there is no protection to those who disclosure of occurrence reports and the disclose through any formal or informal subsequent investigation. Internally, mechanisms. disclosure is limited. Data is disclosed on an ad hoc basis. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 11 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for all levels Just culture should not be seen as an isolated, separate phenomenon within the organisation. It is an outcome of open reporting (a prerequisite for there to be a just culture) and it is part of the organisation’s overall culture, in much the same way as safety culture. Just culture is fundamentally concerned with safety, with the knowledge that is gained from disclosing information about a reporter’s experience and how this is used to derive safety interventions and improvements that lead to more effective system safety. Evidence for a just and ‘open climate’ can be sought in a number of different ways that can assess just culture and its effectiveness. An organisation that has a just and ‘open climate’ will be one that:  emphasises that the purpose of just culture is to gain access to knowledge of the safe functioning of the productive function, and does not place an undue emphasis on ‘gross negligence’;  embraces a reporting and investigation process that recognises the value of the reporter’s experience and the contribution and value that this addition to knowledge brings to the safe and effective provision of services;  emphasises the value of knowledge gained from self-disclosure;  creates an environment where disclosure does not stigmatise individuals and works with peer groups as well as staff representatives to foster a climate of open discussion about experiences freely — reporters will share their experiences to increase the learning potential. To achieve the above a level of confidence and trust is required within the organisation as a whole, but especially within the safety, supervision, managerial and operational actors. This is sustained by engagement, through an active discourse, through a shared belief within and across organisational groups that fairness and the safe provision of services is the objective. Guidance for the Managed Level Just Culture At the ‘Managed’ level, the just-culture principles in an ANSP will have been implemented. For implementation to have taken place, a number of enablers will need to have been established: 1. A just-culture policy will have been developed and adopted. This policy will have evolved through the evolution of a just-culture discourse through engagement between the just-culture decision makers in an ANSP as well as others who can help create the just-culture discourse, e.g. staff associations, professional bodies, supervisory staff. This policy will reconcile how different functions and roles within an organisation understand safety. Understanding the issues and points of conflict between those inside and outside the OPS room and how differences are managed is one indication of the commitment to just culture within the ANSP by managers, safety teams, and staff associations. 2. The deliberations around what is gross negligence, or, more importantly, what satisfies the provisions of Article 16 point 10 of Regulation (EU) No 376/2014 with regard to wilful misconduct and manifest disregard of obvious risks will be explored and discussed with all internal stakeholders, e.g. staff associations. The interpretation of these provisions has consequences. An ANSP that is at the ‘Managed’ level will have developed positions and processes that manage the situations where an occurrence is considered to have breached the thresholds. The context and circumstances of each occurrence will be recognised as being potentially different and thus the different contexts need to be understood. Occurrences will be considered TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 12 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI not solely in terms of what the people closest to the event did, but there will be evidence that wider systemic factors are or will be explored and examined, e.g. training, technical system limitations, procedure under-specification, supervisory decisions, the use of ATFCM, etc. 3. A process will be in place that arbitrates and adjudicates decisions with regard to the determination of what is deemed as acceptable and unacceptable behaviour, especially with reference to Article 16 point 10(a) and (b) of Regulation (EU) No 376/2014. There follows another element of the process where these decisions are. Training and preparation of staff is fundamental to implementing a just culture. The ANSP will have completed some form of initial training, and will then be preparing for or engaged in delivering further training across the organisation to refresh the policy and process as well as to keep the concept of just culture alive. Attendance will have covered managers, supervisors and others involved with administering just culture. Finally, an ANSP that has achieved the ‘Managed’ level will be aware of, and recognise, that just culture is not a component that is ‘stand-alone’ or is and of itself the ‘end point’ of safety within an organisation. Just culture is in the service of safety per se and is an enabler or a necessary part of the evolution to another iteration of system safety. This will be observed in the ANSP’s or organisation’s dialogue around safety. The value of reporting Reporting of occurrences is recognised more as an activity that complies with EU legislation and with the SMS, and a means to enhance safety by deriving a risk landscape driven by a causal-factors scheme. The causal-factor scheme will emphasise risk landscapes that reflect the human as the cause of events. These will be derived by an investigation process that is predominantly focused on ‘what’ happened as opposed to ‘how’. ‘Why’ it happened is expressed in terms of human behaviour, i.e. human performance. Some events will be found to defy explanation in these terms and lead to some investigations being undertaken with an alternative perspective, but it is not pervasive. Options may include: a view of what happened based on second stories; human error as the start of the investigation as human error is a symptom of a deeper problem within the system; a view of what happened from the local rationality of those involved (i.e. why did it make sense to them to do what they did?); and a system’s view of what happened: multiple perspectives from those involved including the wider organisational view. What is ‘acceptable’ and ‘unacceptable behaviour’ will be something, at the ‘Managed’ level, that will still be inferred from the investigation. Examples are micro-matching what people did with the view of what should have been done as defined in rules, procedures, manuals; consequences of individual decisions and actions are emphasised, representing a narrative that is close to the proximity of the event; causal explanations will emphasise the human as cause despite some exploration of the context surrounding the event. There is evidence, therefore, that the limitations of the human-centric approach are recognised and is evidenced in both the investigation narrative and recommendations. The Reporting culture An ANSP at the ‘Managed’ level will use the monthly counts of occurrence reports filed as an indication of open reporting, but will recognise that the nature of the reports filed or the quality of the process is contributing to a reluctance to report by those who are expected to report. In some cases, this may be TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 13 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI directly related to the consequences of just culture. It is known that when consequences (e.g. disciplinary action, retraining, or the application of organisational justice) that emanate from an occurrence report being filed, this can lead to a marked reduction in the level of occurrences reported. The ANSP or the organisation will have put in place occurrence-reporting mechanisms that will support the willingness to report. For example:  feedback to those who report that is timely and meaningful;  growing recognition that those who report have unique knowledge and understanding of the operational situation and event that can contribute to making the operational environment safer and/or more effective;  the reporting processes and methods, and the way that investigations are conducted are consistent with the just-culture policy and principles;  the value of reports is acknowledged and the safety interventions or improvements that flow from reporting are fed back to reporters;  recommendations for safety improvements have ‘owners’ who have the authority to enable the recommendations to be fulfilled. Disclosure Disclosure of safety data to external sources can expose those reporting as well as the organisation too, amongst other things criticism, potentially legal action and unwarranted interference. The disclosure will be undertaken following the development of processes that protect both those who do disclose as well as facilitate the occasions where there is a bona fide reason for disclosure. In some cases, ANSPs may have engaged in discussions with external stakeholders, e.g. NSAs, to establish working arrangements to protect safety data that is disclosed to them or other external bodies that have a legitimate claim to safety data and received assurances through protocols or agreements to protect from unwarranted use of release into the public domain of such data. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 14 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 1: Safety Culture Study Area 1: Development of a Positive and Proactive Organisational Culture Level A Level B Level C Level D Question 1.3 Informal Defined Managed Assured Arrangements Regular assessment of safety culture and The organisation does not see the need to The organisation undertakes a snapshot of The organisation undertakes periodic The organisation undertakes assessments of an improvement programme. have a safety culture assessment mechanism how employees understand safety, in the assessments of safety culture, based on the safety culture. The organisation keeps in in place. belief that this is an assessment of safety organisation’s need. mind the risk of staff disengagement if these culture. assessments are carried out too frequently. No improvement programme is necessary as The assessment method is questionnaire there is no belief that safety culture makes a The assessment method is limited to simple based. The questionnaire is developed using The assessment methodology is multi- contribution to safe production binary questions (such as yes/ no). The the body of knowledge from safety culture faceted. organisation is treated as a single group of studies and includes stratified samples where respondents (it does not recognise sub- different groups are identified and sampled.  Questionnaires are designed cultures). around areas of interest for the Preparation for the assessment is made executive/management as well as The assessment (preparation, collection, data formally including a commitment and what matters to staff. analysis) is conducted in an informal, ad hoc endorsement from the executive  The design of the questionnaire is manner. trialled and involves staff Analysis of the results is undertaken using associations. Analysis of the results is limited to descriptive structured approaches that are able to  The limitations of questionnaire- statistics. contrast the views of different organisational based assessments are resolved by groups and sub-cultures. using focus groups or other such mechanisms. The results are communicated to the wider  This provides an understanding of organisation. the results as well as meaningfully The output of the assessment is used by and purposefully engaging with management in improvement programmes. staff. Analysis of results is structured and explores the differences between different sub- groups/cultures of the organisation. It is also designed to explore the underlying meaning of responses. The emphasis in assessments is to engage and understand what staff have to say. Focus groups are undertaken that use mixed groups of personnel. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 15 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI The results are communicated widely around the organisation, and are discussed with informants, for example through briefings. The output of the assessment is used by management in improvement programmes. These are developed from the results in a collaborative manner with staff and staff associations. The results are benchmarked with external organisations. Guidance for all levels Measuring and assessing safety culture is a practice that allows organisations, if undertaken in a systematic and structured way, to gauge the state and strength of their safety culture and to identify the stressors that are influencing it. There are numerous and varied ways to assess and measure safety culture. All have strengths, weaknesses and limitations. Therefore, organisations that undertake measurement and assessment of the safety culture will need to demonstrate an understanding of these and explain how:  the choice of the assessment method was influenced by consideration of strengths, weaknesses and limitations;  these were considered when analysing and reviewing result data;  these were used to determine the safety culture. One of the most popular instruments for assessing and measuring safety culture is through a ‘Safety Culture’ questionnaire. A safety-culture questionnaire can be defined as a means to conduct a survey that aims to elicit the views and attitudes of respondents about safety in an organisation. These can include values (said and done), beliefs, assumptions, and attitudes towards others. These views and attitudes can be grouped into themes that can be drawn from models of organisational safety culture. There are significant caveats around the use of methods such as questionnaires:  They have been described as ‘quick and dirty’ thus not capturing respondents’ views on safety culture but safety climate.  Questionnaires will not provide the depth required to assess culture — but attitudes are often superficial and transient.  Generalisability of safety-culture questionnaire results have been found to be unreliable.  Unwanted influences on questionnaire respondents cannot be controlled.  Safety climate and safety performance have been found to be weakly correlated.  Culture is by its definition, concerned with shared.  No discrimination between perceptions and attitudes can be undertaken thus obscuring results obtained from a safety-culture survey questionnaire.  Some safety-culture survey methods have added steps that explore the results taken from the survey questionnaire. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 16 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Managed Level For ANSPs at the ‘Managed’ level, a safety-culture assessment will be carried out consistent with the ‘quick and dirty’ administration of safety-culture surveys. The frequency of such assessments will be compliant with ICAO Annex 19 and other documented processes (e.g. provisions of the SMS, included in unit safety plans, or as a follow-up to an earlier safety-culture assessment). The development of the assessment tool for an ANSP will be questionnaire based using both closed and open questions. The questionnaire will be designed in a formal and structured way and will be piloted to calibrate the instrument as well as to assess the scope for misinterpretation of questions and checking the sense of questions. It will target specific groups of staff in the ANSP allowing different views from different groups across the organisation as the basis for understanding different concerns as well as perspectives of safety. The results are openly shared and provide the opportunity to discuss these with senior managers. Before the administration of the questionnaire, there will have been engagement with staff associations for comments and subsequent agreement. The ANSP will have achieved senior management commitment to the safety-culture assessment prior to the administration. This commitment leads to an endorsement and promotion of the safety-culture assessment by managers at all levels. ANSPs at the ‘Managed’ level can be expected to use structured approaches to analyse survey assessment data. The results will be descriptive and will be able to compare and contrast the views of different groups’ answers to the questions. Data from open questions will be found to be of particular use, but no provision will be made for following up the results within the assessment methodology. Where an ANSP has access to statisticians or operational research teams, more sophisticated statistical techniques may be used, e.g. multi-variant techniques, non-discriminant statistics. At this level, ANSPs will not use techniques such as focus groups for follow-up discussions initially preferring to accept the interpretation of the analysis and results, with its acknowledged limitations, by management teams. However, there may be use of meetings where the results are presented back to staff who provide managers with feedback on the results. There will be limited use made of what is learnt. Safety improvements and interventions are driven and constructed by managerial teams. However, the experience and what is learnt from the assessment is seen as a valuable source of knowledge of the state of the organisation. This is a catalyst for change in the safety discourse as well as its acceptance as a tool for managerial action to improve efficiency of operations and safety. The nature of the resultant safety interventions and improvements will be naturally superficial and very few if any that lead to changes within the operational environment. Results are published and fed back within the organisation but not shared externally. Guidance for the Assured Level At the ‘Assured’ level, the ANSP will be attentive to the issue of staff disengagement if these assessments are too frequent and recognise that the frequent administration of the survey method does not allow interventions and improvements to have full effect such that it will change the respondents’ perceptions and attitudes. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 17 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 2: Safety Policy Level A Level B Level C Level D Question 2.1 Informal Defined Managed Assured Arrangements The safety policy of the organisation presents The need for a safety policy has been The organisation has drafted a safety policy. The safety policy has been signed by the The safety policy is subject to ongoing the organisation’s commitment to both safety recognised but one does not exist. most senior manager in the organisation review and improvement (e.g., when a new and its resourcing. The priority of safety The draft safety policy is available for review (e.g. CEO) and has been formally published. executive becomes accountable for safety within the organisation is also articulated. within the organisation. or when there are indications that the policy The organisation conducts reviews of its does not adequately address the adequate The safety policy reflects the priority of safety policy at least once every five years level of commitment to safety). safety in the organisation. to ensure that it continues to be relevant and appropriate. The organisation compares its safety policy to those of other ANSPs. The organisation has sufficient staff and resources to implement its safety policy and If changes are made to safety policy, the related procedures. organisation has a process to ensure that the SMS is updated to meet the amended The safety policy has been communicated to requirements of the policy. employees throughout the organisation. Updates to the safety policy are communicated throughout the organisation. Guidance for the Managed Level The safety policy is formally published, either internally or externally, as appropriate and in accordance with the organisation’s SMS. The concept of reviews is an ICAO requirement and good practice would be to publish internally, as a minimum, using local mechanisms and ensure that staff are aware of the policy and how to access it. There is a defined period of review within the organisation’s safety policy. Everyone understands the role they play in delivering operational safety performance and they have the capability to discharge their role. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 18 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 2: Safety Policy Level A Level B Level C Level D Question 2.2 Informal Defined Managed Assured Arrangements The safety policy addresses key attributes of The organisation is considering which key The organisation's approach to safety is There is a clear relationship between the The organisation conducts periodic reviews the organisation’s approach to safety. These attributes of its approach to safety should reflected in its developing safety policy or organisation's safety policy and its SMS. of its approach to safety management and, attributes will most likely include culture, be included in its safety policy. related procedures. where necessary, updates its safety policy visible endorsement, communication and The organisation's safety policy or related and related procedures. safety reporting. procedures determine how safety management is implemented throughout the organisation. The organisation's safety policy or related procedures define the procedures for safety reporting, including the types of behaviours that are acceptable and the specific circumstances under which disciplinary action might apply. Guidance for the Managed Level Safety policy is used to set safety accountabilities for senior management. There is a clear relationship between the safety policy a nd the procedures in the SMS. With respect to disciplinary actions, organisations need to consider the impact of such disciplinary actions on establishing and maintaining a just and open reporting culture. They should consider the protections afforded by Regulation (EU) No 376/2014, and specifically Article 16, points 9 and 10. Organisations need to clearly state in their safety policy the circumstances and reasons why actions might be considered to fall within t he scope of point 10. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 19 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 3: Safety Accountabilities Level A Level B Level C Level D Question 3.1 Informal Defined Managed Assured Arrangements An approved, clearly documented, and No formal designation of responsibilities, The organisation has identified its safety The organisation has defined and The organisation reviews safety authorities, recognised system for the management of accountabilities or authorities for the responsibilities, accountabilities and documented authorities, responsibilities responsibilities and accountabilities at least safety. Management structure, management of safety exists. authorities. and accountabilities for safety management. once every five years to determine whether responsibilities, accountabilities and they are suitable and effective. authorities are clearly defined and Line managers accept responsibility for The organisation has an accountable documented. management of safety. executive who has ultimate responsibility for the management of the SMS. The wider leadership team takes responsibility for the application of the SMS. The organisation reviews safety responsibilities after significant organisational changes. Guidance for the Defined Level Line management is usually responsible for the implementation of procedures or practices which are required by the SMS, with specific responsibility for the development and application of the SMS. These responsibilities are not yet formally defined. Guidance for the Managed Level The wider leadership team is the team of people who report directly to the accountable executive. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 20 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 3: Safety Accountabilities Level A Level B Level C Level D Question 3.2 Informal Defined Managed Assured Arrangements A clearly defined safety management function / A safety management function within the The organisation has a safety management The safety management function or Leadership, at the highest level, recognises safety manager that is independent of line organisation has not yet been formed to function or safety management position position is independent of operational line its role in the SMS and actively supports its management. develop the SMS. responsible for developing and management. development, implementation, maintaining the SMS. maintenance and promotion throughout The safety management function or the organisation (including support position has the authority to develop and departments). maintain an effective SMS. The safety management function or position has access to the resources required for the proper development and maintenance of the SMS. Guidance for the Managed Level The safety management function or position independence of operational line management means that it reports and is accountab le directly to the highest organisational level. Guidance for the Assured Level Safety leadership — the head of the organisation and senior management have made a commitment to safety and its application by fostering a just culture throughout the organisation. Air traffic service providers (ATSPs) should determine their own safety responsibility set as appropriate to their individual organisations. Individual job titles may differ from one organisation to another, an example of this foll ows: The head of the organisation might be termed the ‘Chief Executive’, and the safety manager might be called the ‘Safety Director’. Support departments are intrinsic parts of an organisation; while not directly being part of the ATS provider, are involved in day-to-day operations in providing safe operations, e.g. CNS, MET, AIS. The SMS is an effective management system which assists decision-making at the very highest levels. The executive board is actively involved into safety-promotion activities. . TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 21 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 3: Safety Accountabilities Level A Level B Level C Level D Question 3.3 Informal Defined Managed Assured Arrangements Safety management accountabilities and Knowledge of the principles underpinning Relevant staff and contractors apply rules Relevant staff and contractors are aware of Relevant staff and contractors throughout responsibilities are understood clearly and SMS among all staff and contractors is and procedures to their tasks. how their actions affect the safety of the the organisation have responsibility for accepted by all relevant staff and contracted negligible. wider operation. promoting and improving safety. staff. Relevant staff and contractors are at least partially aware of their roles and Relevant staff and contractors are aware of The organisation reviews and assesses accountabilities in the SMS. how the actions of others affect safety. documented safety management responsibilities at least once every five Accountability for safety in the years. organisation is understood by all relevant staff and contractors. Relevant staff and contractors take proactive, day-to-day action to have rules and procedures changed where they identify a potential safety benefit. Guidance for the Defined Level Relevant staff and contractors are those whose activities can impact on the safety of operations. Relevant contractors are those who are required to apply the organisation’s SMS. For example, in the case of contracted staff that clean the OPS room, the supervisor would have accountability for ensuring the staff are appropriately briefed. The staff themselves would not have the accountability. Guidance for the Assured Level Staff and contractors believe that it is their responsibility to take action to have rules and procedures changed where they identify a potential safety benefit. The documented safety management responsibilities are the responsibility of the safety manager and probably need to be endorsed by a safety review board (SRB). The internal SRB provides internal governance for the organisation. The members of the SRB are typically the senior managers accountable for the safety of the organisation. This SRB will, for example:  assure that safety risks and safety issues are proactively identified and effectively managed;  measure safety performance against safety targets and assure that appropriate action is taken ; TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 22 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI  assure that safety improvement actions across the organisation are prioritised and coordinated effectively, and that responsib ility for follow- up action is allocated;  own and support SMS development; specifically, review safety policy at least once every 3 years, taking into account best safety practices in similar industries;  provide direction for the continuous improvement of safety, including the recognition of best practices and implementation of lessons learned from internal and external sources;  assure that the safety accountability and responsibilities of the head of the organisation are reviewed regularly and maintain ed;  coordinate and track actions and recommendations arising from the Safety Oversight . TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 23 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 4: Coordination Emergency Response Plan Level A Level B Level C Level D Question 4.1 Informal Defined Managed Assured Arrangements Emergency response procedures and an The organisation has sound primary air The organisation has procedures and at least Emergency response procedures have The organisation regularly exercises its emergency response plan that documents the traffic management systems but does not some redundant capabilities and resources been developed, documented and emergency response procedures at least orderly and efficient transition from normal to have redundant capabilities or back-up to manage abnormal and unexpected distributed to the appropriate staff. once a year and regularly revises its emergency operations and the return to normal systems situations. emergency response plan. operations. The organisation rehearses emergency response procedures at least once per The organisation's emergency response year. procedures have been rehearsed through live or simulated exercises at least once The organisation's emergency response in the past three years. plan been properly coordinated with the emergency response plans of other The organisation uses indicators to organisations that it must interface with assess the effectiveness of its emergency during the provision of services (ICAO response procedures, as tested during Annex 11 – 1.4). the regular exercises and rehearsals. Guidance for the Defined Level There are procedures and resources to cope with abnormal and unexpected situations. Guidance for the Managed Level The organisation ensures that emergency response procedures are updated at least once per year, e.g. contact information. To achieve the managed process, organisations should have a defined and documented process that has been shown to work. Emergencies include sudden system failures or other abnormal or unexpected situations, such as:  the loss of major air traffic systems, (e.g. radar display picture, electronic flight progress strip system, standby and emergency communications on multiple frequencies due to external interference);  the loss or failure in support facilities (e.g. power, air conditioning, building integrity);  aircraft emergencies (e.g. emergency descent, hijack, air defence security);  disruption of air traffic services (e.g. emergency dispersal of traffic, closure of an adjacent air traffic centre, runway closure leadi ng to mass diversion). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 24 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI The ‘plan’ should encompass what is to be done, including the interactions with other organisations (e.g. police, emergency services) and the ‘procedure’ should describe how it is to be done. See requirement ATS.OR.200(1)(iv). For example, Letters of Agreement or any other form of service agreement are in place with organisations and support the emergency response plan. Guidance for the Assured Level To reach the ‘Assured’ level, the organisation should be able to measure the output by running a simulation. The simulated exercise may include, for example, aircraft accident, hijacking events, environmental disaster, access to the OPS room, bomb threat, etc. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 25 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 5: SMS Documentation Level A Level B Level C Level D Question 5.1 Informal Defined Managed Assured Arrangements A formal SMS that meets all applicable There is no SMS in place. There may be The organisation has started to implement The organisation's SMS meets all safety The organisation exceeds minimum safety and regulatory requirements. deviations from safety regulatory its SMS. regulatory requirements. compliance requirements by operating at a requirements. higher standard of safety management. The organisation has developed an The organisation has completed all work The need for an SMS implementation plan is implementation plan to ensure that its SMS required by its SMS implementation plan. recognised. will meet regulatory requirements. Guidance for the Managed Level There is a defined function responsible for ensuring that the SMS continues to meet regulatory requirements. There is a document in th e SMS that maps the SMS against current regulatory requirements and shows that those requirements have been satisfied. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 26 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 5: SMS Documentation Level A Level B Level C Level D Question 5.2 Informal Defined Managed Assured Arrangements Clearly defined and documented safety Operations manuals do not contain specific The SMS implementation plan includes SMS is implemented. The organisation monitors its SMS processes standards and processes. safety management procedures. requirements for: and outputs regularly to identify any Safety management is documentation is problems employees may have in applying  Safety policy and objectives readily available to appropriate staff. the SMS.  SMS requirements  SMS processes and procedures  Accountabilities, responsibilities and authorities  Outputs such as investigation reports, performance trend reports and safety documentation to support changes to service delivery Guidance for the Managed Level The organisation has published the necessary procedures, processes (e.g. SMS policy/framework) and tools (e.g. collecting hazards/deficiencies, feedback, lesson dissemination). Guidance for the Assured Level Evidence shows that relevant SMS processes and outputs (at least safety policy, SMM, occurrence reporting and in vestigation procedures) are reviewed on an annual basis (e.g. internal audits, peer review, safety board meetings). Verification results are documented and measures taken without delay where there is a safety impact. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 27 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 2: Safety Policy and Objectives Study Area 5: SMS Documentation Level A Level B Level C Level D Question 5.3 Informal Defined Managed Assured Arrangements Safety management documents are There is no formal process that maintains the The organisation has an informal process to The organisation has a formal process for The organisation conducts formal reviews of regularly reviewed, assessed and SMS, nor is there an identified authority (or address amendments to its SMS. maintaining all safety management processes any organisational changes that could affect maintained. authorities) responsible for the updates. and procedures. safety and/or the safety management Someone within the organisation is framework. responsible for updating the SMS. The organisation's SMS is regularly reviewed and updated. The organisation assesses the usability and accessibility of its SMS processes and documents. Guidance for the Managed Level ‘Formal process’ means that the description of the responsibilities, input, output, activities, etc., put in place by the organisation for maintaining its safety management processes and procedures is formalised (documented) in the SMS documentation and is up to date. ‘Regularly reviewed’ means that the SMS is reviewed and, if needed, updated at least at the following occasions:  whenever there is an organisational change or a change in the provision of services that can have in impact on the SMS;  when analysing the outcomes of the safety monitoring system and SMS audits; and in any case every 5 years (in line with point 14.1). Guidance for the Assured Level The types of justifications include the following:  evidence and/or outputs stemming from the formal review process;  feedback on its SMS processes and documents from staff working within the SMS procedures. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 28 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 3: Safety Risk Management Study Area 7: Risk Management Process Level A Level B Level C Level D Question 7.1 Informal Defined Managed Assured Arrangements Hazards to operations are reported and Hazards to operations are not highlighted by The organisation is developing processes to The organisation has a sufficient number of The organisation reviews and updates its assessed. either managers or staff. However, risks to assist in the identification and reporting of qualified employees to assist in identifying hazard identification and analysis processes operations are recognised. hazards. and assessing hazards. at least once every five years. The organisation is developing processes to The organisation has taken reasonable steps The organisation monitors whether the assess the risk that hazards pose to to identify all hazards affecting its operations. hazard identification process is appropriately operations. applied. The organisation's hazard identification The organisation is developing processes to process is based on a combination of document the existence of hazards and their reactive, proactive and predictive methods of risk levels. safety data collection. The organisation regularly includes stakeholders in its identification and assessment processes. The organisation addresses identified hazards as part of its process to improve safety performance. Guidance for the Defined Level Processes to assist in the identification and reporting of hazards and the level of risk posed to operations may be, for example, covered by the Operational Unit Safety Case, which are used to assess and manage the risks posed by hazards to operations. The Operational Unit Safety Case is documenting the hazards and level of risks. Guidance for the Managed Level To identify threats, an ANSP should present a range of risk/hazard identification techniques to assist staff in identifying p otentially unsafe events. In simple terms, this means determining what events can happen and when, where and why. There are a range of techniques that can be used to determine these elements. The technique used will depend upon the scenario under development and the life cycle stage at which the risk management activity is being undertaken. The organisation ensures that it dedicates sufficient resources to assist in the identification and assessment of hazards, and that these staff are adequately trained in efficient techniques to identify and assess hazards and their risks. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 29 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI These techniques of hazard identification includes combination of reactive, proactive and predictive safety data collection and measurement. Lagging indicators are reactive measures whereas leading indicators are proactive measures:  Reactive: mitigate severity of safety events and threats;  Proactive: identify safety concerns before safety events happen; and  Predictive: inputs to and outputs from the safety system are used to predict future outcomes, and anticipate future exposure based on past performance data. See Regulation (EU) 2017/373, and GM1 ATS.OR.200(3)(i). The organisation involves all relevant stakeholders in the hazard identification and assessment process, including internal (e.g. operational staff) and external stakeholders (e.g. users of its ATC services or providers of services used in the provision of ATC services) setting up multidisciplinary teams, when needed. Guidance for the Assured Level Given the central role that risk management plays in an ANSP’s SMS, it is essential that practices, processes, tools and poli cy are monitored and improved or updated as necessary. Such continuous improvement is supported by an effective review and monitoring cycle that may includ e the following: • measure risk management performance against established indicators; • measure progress against the goals set in the Risk Management Implementation Plan; • review the framework in light of internal experience and external benchmarking; • expand risk techniques based on industry experience (e.g. adopt the barrier model); • test compliance with the requirements of the risk management process; • report on how effective the organisation has been in meeting the objectives described in its risk management and safety polici es. Emerging risks may include drone operations, commercial space launches, etc. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 30 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 3: Safety Risk Management Study Area 7: Risk Management Process Level A Level B Level C Level D Question 7.2 Informal Defined Managed Assured Arrangements Assessed risks are mitigated or controlled. There is limited understanding of the need The organisation acknowledges the need to The level of analysis, assessment, The organisation reviews the level of risk it to mitigate or control risk, even when risks mitigate and control risks. mitigation and control of risk being can accept at least once every five years on are recognised. undertaken is proportionate to the severity the basis of its performance. The organisation has proposed the level of of the risk outcome. risk that individual managers can approve. The organisation reviews its level of risk to The organisation documents and enforces ensure it is in line with the risk tolerance The organisation is establishing processes the level of risk that its managers can level of its governing body (e.g., Board). to document how appropriate controls and accept. mitigations should be selected. Guidance for the Defined Level The organisation is establishing processes to document how appropriate controls and mitigations should be selected, for example, through the hazard identification process. Controls are preventative mitigations and/or recovery mitigations. Guidance for the Managed Level This level of risk that can be approved is documented. When an individual or organisation accepts a risk, it does not mean that the risk is eliminated (i.e. some level of risk always remains, called residual risk). Rather, the individual or organisation accepts that the residual risk is sufficiently low. There is less demanding process for analysis, assessment, mitigation and control when the resulting risk is minor. The organisation ensures that managers can only accept risk levels that have been determined and documented. Guidance for the Assured Level The organisation uses actual operational performance data to review its risk criteria based on its safety performance. To ach ieve this level, at least 5 years of performance data are required to be used in the review. This level of risk is ensured to be in line with the risk-tolerance level defined for the safety board of the organisation. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 31 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 3: Safety Risk Management Study Area 7: Risk Management Process Level A Level B Level C Level D Question 7.3 Informal Defined Managed Assured Arrangements Risk controls are monitored for effectiveness, There is little understanding of what There is a reasonable understanding of risk The organisation has formally documented The organisation regularly monitors the and remedial action is taken if controls are not constitutes a risk control at either a system controls in the organisation. its risk control processes. effectiveness of risk controls. working effectively. or local level. The organisation is developing processes to The organisation is implementing risk Where deviations or deficiencies are The effectiveness of these controls is not identify, assess and control operational control processes. identified, the organisation has proposed evaluated. risks. improvements to the risk control The organisation is identifying and framework. documenting operational risk controls. The organisation’s long-term investment The organisation has implemented programme provides for improvements in processes and practices that allow it to safety that address key risks (e.g., safety measure its operational risk baseline. tools, additional staff, training). The organisation identifies and manages performance deviations and deficiencies from its operational risk baseline Guidance for the Defined Level Risk controls are new components or modifications of the existing ones. These components are of different nature, e.g. proced ures, technological (either software or hardware), training, etc. In other words, risk controls are changes to the functional system, or part of a wider change to the functional system, aiming to control safety risks that have been identified by the ATS organisation. Guidance for the Managed Level The ATS organisation has to develop risk-control processes to identify, assess and control safety risks. These processes will be documented as part of its SMS processes, and the organisation will effectively apply them. These processes may be embedded in the wider processes of monitoring the behaviour of its functional system within its context of operation and the management of changes to the functional system of the ATS organisation. They will aim to identify, manage and mitigate associated risks to the behaviour of the ATS service in the context where it is provided and to any change to the functional system that is proposed for implementation, to an acceptable level, as appropriate, by using specifi c and verifiable safety criteria. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 32 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI The resulting risk controls need to be clearly identified and documented to allow a proper monitoring of their effectiveness. The monitoring of the effectiveness of the risk controls will involve definition of indicators of the safety performance of the behaviour of the ATS service, and the specification of the monitoring criteria necessary to demonstrate that the service delivered by the changed functional system , after the implementation of the risk controls, will continue to meet the safety criteria. These indicators associated to the safety criteria can be represented totally by safety risks, by other measures that relate to safety risk (i.e. proxies), or a mixture of safety risks and these other measures. Therefore, the processes and practices that allow it to measure its operational risk baseline should be understood as those processes and practices that allow the organisation to set safety criteria relating to changes to the functional system and design effective indicato rs that represent the safety criteria in terms of risks, proxies, or a mixture of both, and the specifications of the monitoring criteria to ensure that t he safety criteria are continuously satisfied after the implementation of the changed functional system containing the identified risk controls. The operational risk baseline is determined by measuring a series of lagging indicators on the performance of the ATSP. The documented risk controls, the measurement of the indicators related to the associated safety criteria, and the monitoring criteria will render achievement of the ‘Managed’ level. Guidance for the Assured Level When these risk controls (and associated indicators) are monitored periodically, the level ‘Achieved’ will be ‘Assured’. The organisation should be able to demonstrate when was the last time that the review took place, and that it was in line with the stated periodicity. Deviations or deficiencies identified in the monitoring should be part of the risk-control process, and it should trigger changes to the risk controls. This means that the risk-control process should include a process to develop corrective actions, i.e. further changes to the functional system. There is a formal responsible within the organisation to ensure improvements in the risk-control framework. There is a corrective-action procedure that monitors performance deviations and deficiencies from its operational risk baseline. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 33 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 11: Safety Reporting Level A Level B Level C Level D Question 11.1 Informal Defined Managed Assured Arrangements A continuing organisation-wide process to There is an informal system in place for The organisation investigates incidents, The organisation has a formal reporting and The organisation checks to ensure that all report and investigate safety occurrences and reporting safety occurrences, but reports even if there is no formal investigation investigation system. required occurrences have been reported. risks. are not reviewed systematically. process. The organisation keeps formal records of all The organisation monitors the number of The reporting system is not organisation- The organisation provides feedback to staff incident and accident reports and related reports that require investigation but are wide. on investigation findings. information. yet to be investigated. Investigation is done on an ad hoc basis with Investigations result, if necessary, in The organisation measures the quality and little or no feedback. corrective and preventive action. effectiveness of its investigations. Staff reporting safety occurrences can also suggest ways to solve problems identified in their occurrence reports. The organisation provides feedback to those who report occurrences or hazards of any corrective actions taken as a result of their report. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 34 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Defined Level The feedback provided to staff in the ‘Defined’ level is of general nature and it is done on an ad hoc basis. Guidance for the Managed Level The organisation has a formal reporting and investigation system, including both mandatory and voluntary occurrences. The dis tinction between mandatory and voluntary reports is given in Regulation (EU) No 376/2014. There is a formal process in place to ensure that corrective and preventive actions are monitored and managed. The occurrences and related investigation information is recorded and personal data are secured. De -identified information can be disseminated within the organisation, as required. Personal details are protected and only used to investigate occurrences with a view to enhanc ing safety. Staff are allowed, and even encouraged, to provide solutions either during the initial reporting or during the incident interview, as appropriate. The occurrence-reporting system has formal ways to provide feedback to occurrence reporters, as a minimum, either with the result of investi gations or corrective actions to be implemented. Guidance for the Assured Level The organisation actively reminds staff and promotes the reporting of occurrences, either by safety-promotion campaigns, surveys and/or audits that emphasise the importance of occurrence reporting. The organisation measures the quality and effectiveness of its investigation process. This concerns more the quality of the process, and less to the effectiveness of the investigation output. In particular, the number of open occurrences that require investigation, thereby monitoring the time taken to close the investigation. Good practices include, for example, to apply a moderation process to ensure consistency of the investigations and that the data are recorded, stored, and are of adequate quality and available for future analysis. Notifications on relevant ATM/ANS-related occurrences that have been reported by other organisations (e.g. operators/pilots) are included in the investigation process of the ATS provider. They are used for random testing that these occurrences are reported internally by its staff. Where available, automated safety data recording systems are applied and information used in the identification and investigation of occurrences. The quality of the investigation process is reviewed in the course of internal audits, surveys and peer -review meetings (e.g. safety experts from adjacent ANSPs). The results from external oversight activities are used in order to improve not only the quality but also the effec tiveness of the investigation process. Safety-promotion activities (e.g. briefings, safety days, leaflets in the OPS room) focusing on mandatory occurrences are conducted regularly. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 35 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Element 12: Safety Surveys and Audits Level A Level B Level C Level D Study Area 12.1 Informal Defined Managed Assured Arrangements Internal and independent (external) safety There is no plan to conduct systematic The organisation has a plan either in place The organisation has a formal process The organisation carries out safety surveys surveys and SMS audits. safety surveys and SMS audits. or under development to formalise how describing how to conduct SMS audits. in addition to SMS Audits. SMS audits are conducted. Safety surveys, SMS audits, and gap The organisation conducts internal SMS The organisation's safety surveys are assessments are conducted on an ad hoc The organisation has carried out any SMS audits at least annually. carried out systematically. basis. audits. SMS audits have resulted in the The organisation has established a process development and implementation of to analyse trends arising from safety improvement plans. surveys and SMS audits. Where appropriate, the organisation conducts reassessments to confirm that any implemented recommendations arising from safety surveys and SMS audits have been successful. The organisation commissions external surveys and SMS audits at least once every five years. The outputs from safety surveys and SMS audits are incorporated (as appropriate) into operations or the SMS. The organisation has established a process that requires external data (e.g., pilot non- conformance with ATC instruction trend information) to be considered when selecting topics for operational safety surveys and SMS audits. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 36 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Assured Level Safety audits focus on the integrity/compliance of the entire SMS whereas safety surveys proactively concentrate on particular elements of the SMS or procedures of specific operations (e.g. problem areas, areas of confusion). The surveys are used to identify ‘what goes ri ght’ and ‘what needs to improve’. Safety surveys provide a systematic review to recommend improvements where needed, to provide assurance of the safety of current activities, and to confirm conformance with applicable parts of the SMS. During safety surveys, auditors examine procedures or processes related to a specific operation to identify weaknes ses and/or areas for safety improvement within the aviation service provider’s organisation. Safety surveys are conducted on the basis of a safety survey plan. The safety survey’s results are documented in a survey report that also includes the actions to be taken. Lessons learned from safety surveys are disseminated and the actions identified are carried out within the defined time frame. The follow-up is conducted in a systematic way; in addition, the organisation is aware to what extent the lessons learned drive changes into the SMS. External surveys and SMS audits are carried out by an independent body (e.g. EUROCONTROL, SMS experts, competent personnel fr om other ANSPs). The topics for safety surveys and SMS audits may be identified by means of safety performance (e.g. indicators, trends) as well as through suggestions from members of staff and occurrence notifications from different reporters/reporting entities (e.g. ATCOs, pilots, aerodrome personnel, operators). A risk-based approach can be applied if deemed necessary. Data gathered in the course of meetings (e.g. between ANSPs and operators, international best-practice exchange) may also be used to trigger a safety survey or SMS audit. External data could also be gained from stakeholders’ ‘complaints’. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 37 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-010(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 13: Safety Performance Monitoring Level A Level B Level C Level D Question 13.1 Informal Defined Managed Assured Arrangements An established and active monitoring system There are no indicators, thresholds or The organisation has a plan in place or The organisation has implemented and The organisation has developed targets to that uses and tracks suitable safety indicators formal monitoring systems in place to under development to implement a safety formally documented a safety performance reflect its safety policy and risk tolerance. and associated targets (e.g., lagging and measure safety achievements and trends. performance monitoring system. monitoring system. leading indicators). The organisation analyses trends for safety The organisation has established safety The organisation's safety performance improvement purposes. indicators. targets are meeting all applicable regulatory requirements. The organisation has safety indicators covering all aspects of the system or operation. The organisation uses leading indicators to increase the range of safety metrics for measuring its performance. Safety management processes require that any negative trends in safety performance indicators be addressed. Guidance for the Managed Level Targets should not be set arbitrarily. Consideration needs to be given to: (a) the variation and sensitivities in the data monitored thus far; (b) the potential impact of system changes; and (c) the forecast impact of planned safety improvements. Set a realistic target that is based on the organisation’s current performance with reference to previous performance, which results in a long -term view for the organisation. Indicators and targets have been set limited to meeting the safety regulatory requirements to verify the safety performance of the organisation. Statistical measures can be used to identify trends. Consideration also needs to be given to metrics that are not changing wh en they should be, or to risks that may be reducing but not quickly enough. Safety indicators need to cover the full scope of the organisation’s operation and should consider all aspects of the SMS. Ex amples of safety indicators cover a range of metrics, such as safety incidents and associated risk monitoring, surveys, staff atten dance and sickness, or implementation rates of safety policies, procedures and equipment. Leading indicators are early-warning measures that detect a change in the risk levels. For example, falling staffing levels can for a time be absorbed by longer working hours, but eventually more tired controllers may be more error prone. Hence monitoring staffing levels can provide an indication of a change in the risk levels. Other examples include organisational finances and staff sickness rates. Safety improvements may also be forecast via TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 38 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-010(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI implementation of new safety equipment — although care needs to be taken to include the risks associated with the change and to avoid being too optimistic. Either positive or negative, changes in the trends or outputs of performance indicators should be investigated an d understood. Examples of leading indicators of safety may include:  sickness levels,  staffing levels,  staff turnover,  critical incident SM,  workload measures,  failure to comply with regulations,  unmitigated high-level risks,  observational methods (normal operation safety surveys). Guidance for the Assured Level A mature safety performance monitoring system will contain the following elements: monitoring, filtering, trend identification, analysis, mitigation- measure development, dissemination, verification, document, and feedforward. Note that ‘feedforward’ is meant to use leading indicators that put emphasis on anticipated or expected disturbances associated to risks as opposed to feedback that focuses, instead, on actual outcomes. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 39 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 13: Safety Performance Monitoring Level A Level B Level C Level D Question 13.2 Informal Defined Managed Assured Arrangements Methods to measure safety performance, which Ad hoc safety performance data related to At least some parts of the organisation The organisation has implemented The organisation conducts internal is compared within and across ANSPs. individual incidents is available, but there is have implemented safety performance qualitative techniques to measure safety comparative analysis. no systematic approach for measuring measurement processes. performance (e.g., opinion surveys, safety performance. observational techniques, and overload The organisation works with stakeholders reports). to conduct external comparative analysis. The organisation has implemented The results of the organisation's safety quantitative techniques to measure and performance activities influence the verify safety performance. operational safety survey and SMS auditing programme. The organisation has implemented measures to validate the effectiveness of risk controls and mitigations. Guidance for the Managed Level Quantitative techniques should be more than counting events. They are used to establish genuine changes in the system performance and in implementing safety interventions, where appropriate. The organisation uses occurrence reports and investigation reports when evaluating the effectiveness of risk controls and mitigations. Due consideration is given to the contributing factors identified in occurrences and investigation reports. Guidance for the Assured Level Both internal and external comparisons of safety performance should be carefully designed to ensure that differences in the nature and size of the operation, or the data-collection methodologies, are identified and accounted for. Organisations should be proactive in identifying partner organisations with which to conduct comparative analyses. In addition to high-level comparisons, these partnerships may include monitoring where a particular unit or type of operation has no internal equivalent and thus is compared with a unit or type of operation at the other organisation. The aim of such comparisons is to highlight differences that should be examined in more detail in order to unde rstand their causes. The staff should have the opportunity to review the comparative performance analysis of their unit. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 40 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 14: Management of Change Level A Level B Level C Level D Question 14.1 Informal Defined Managed Assured Arrangements Documentation and reporting mechanisms are There are no change management The organisation is developing change The organisation's change management The organisation's change management in place to ensure that internal and external processes in place even though the management processes to assess and processes determine whether a change processes are reviewed and updated at stakeholders understand how safety risks organisation recognises that impacts of quantify the risks of change. should be authorised. least every five years (e.g., from internal introduced during and/or following change must be managed. experience, external lessons learnt). implementation of change are managed and The organisation is developing change The organisation's stakeholders, including mitigated. management processes that require the its regulator, are aware of these processes The organisation assesses the performance involvement of stakeholders. and their purpose. of its risk controls and mitigations as part of its change management processes. The organisation assesses the safety impact of changes and associated mitigations The organisation's change management before they are introduced. processes define and report transitional risk. The organisation's change management processes involve all relevant internal stakeholders. The change management processes are tailored for the importance and the resources needed for the change. Guidance for the Defined Level The organisation does inform other organisations and, where feasible, stakeholders affected by the planned change. Furthermor e, the organisation and these other organisations, in coordination, shall determine: (1) the dependencies with each other and, wher e feasible, with the affected stakeholders; and (2) the assumptions and risk mitigations that relate to more than one organisation or stakeholder. Guidance for the Managed Level The organisation’s change management processes consider the changes to functional systems, i.e. a combination of procedures, human resources and equipment, including hardware and software, organised to perform a function within the context of ATM/ANS and other ATM n etwork functions. The organisation should not start the implementation of any part of the change that has the potential to affect the safety of the services until a valid assessment for that part of the change exists and, if applicable, it has been authorised by the regulator. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 41 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Assured Level A total system approach to the management of change is employed. The ATM system is considered as a whole rather than focusing on the human element. There is a strong relationship between in-service monitoring and design. Change assessments employ a common set of operational hazards and they are monitored in service to confirm the effectiveness of the risk controls and mitigations. Besides, monitoring criteria tailored to the change implemented are part of the change management processes. These criteria are specific to each change and hence ensure that the change will remain acceptably safe for as long as it is in operation. Transitional risks are risks linked to the transition from the current functional system to the changed functional system. Th ese might be mitigated, e.g., by training depending on the nature of the change and the transitional risk associated to it. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 42 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 15: Continual Improvement of the SMS Level A Level B Level C Level D Question 15.1 Informal Defined Managed Assured Arrangements An integrated planning process drives the An ad hoc or non-existent safety planning The organisation is preparing to develop a The organisation has established formal The organisation has a plan to improve the continual improvement of the SMS. process is utilised by the organisation. plan to show how it will improve the planning processes to drive improvement of management of safety risks. Safety goals and objectives have not been implementation and management of safety. its SMS. identified or documented for the The organisation's plan to improve its SMS implementation of an SMS. The organisation regularly evaluates the includes measurable safety management effectiveness of these planning processes. goals and targets. Guidance for the Managed Level The formal planning process to drive improvement of its SMS is aligned with other business planning processes. The organisati on aligns its planning with the SSP and EPAS actions related to the improvement to its SMS, if any. The results of the evaluation on the effectiveness of the planning processes are documented. Guidance for the Assured Level The improvement plan includes the key risks and high-level mitigations; trend analysis of safety data is used when identifying these key risks. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 43 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 4: Safety Assurance Study Area 15: Continual Improvement of the SMS Level A Level B Level C Level D Question 15.2 Informal Defined Managed Assured Arrangements A structured approach to gather and share There is no structured approach to gather The organisation gathers information on The organisation has formal processes in The organisation reviews, assesses, and information on operational safety and SMS best practices from the industry. The operational safety and SMS. place to identify best practices from adopts industry best practices. best practices from the industry. organisation can identify and adopt throughout the industry that can be used to industry best practices on an ad hoc basis. The organisation gathers information on improve the SMS. The organisation has carried out an impact internal best practices to improve safety assessment to determine whether the best management. The organisation shares its best practices practices have been effective in improving with industry stakeholders (e.g., ANSPs, safety. There are no plans to release and share best airlines, regulators). practices with industry stakeholders. Guidance for the Managed Level A mechanism has been established on how to share best practices with industry stakeholders (not limited to aviation stakeholders), and it may include health care, nuclear, etc. Guidance for the Assured Level A best practice is one that proves to be more effective and efficient in producing positive results in terms of safety management. Best practices are determined through peer review by a number of organisations. The organisation has established a formal process to review and assess industry best practices. The impact assessment should be evidence based and adaptable to the organisation’s SMS. The results of the impact assessment are documented. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 44 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 16: Training and Education Level A Level B Level C Level D Question 16.1 Informal Defined Managed Assured Arrangements Staff, and contractors where appropriate, are Staff, and contractors where appropriate, The organisation regularly provides staff The organisation has an annual planning Those who receive training are given an educated and trained in safety and safety are provided with training for safety and and contractors with training and education process for safety management training. opportunity to provide feedback on the management, and where required, licensed. safety management activities on an ad hoc in safety and safety management. effectiveness of the training. basis. The organisation's annual training plan The organisation provides staff and ensures that appropriate staff are aware of The organisation's training programmes are contractors with training and education to all safety management practices and updated on the basis of that feedback. help them apply required safety procedures that are applicable to their management practices and procedures. roles. The organisation uses indicators to measure the effectiveness of its training programme. The organisation's annual training plan ensures that staff are aware of the The training is adapted to include identified organisation's approach to safety. risks and address shortcomings (highlighted through, for example, feedback from courses). Guidance for all levels This objective is primarily focused on ATC, engineering and senior staff who have the ability to affect the safety of the ope rational service. Contractors should receive safety training when their activities have an impact on the provision of (provider’s) services. The safety training should be appropriate to the safety responsibilities of the individual. See requirement ATM/ANS.OR.B.005(a)(6) and ATS.OR.200(4)(i). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 45 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 16: Training and Education Level A Level B Level C Level D Question 16.2 Informal Defined Managed Assured Arrangements Staff are competent to conduct their There are no formal competency methods The organisation is developing competency Competency methods are designed and The means by which competency standards obligations under the SMS. (including proficiency, licensing and methods. applied to ensure that staff, where are determined is subject to review and training) appropriate, are educated, trained and improvement. competent to perform the specific duties required of them by the organisation's SMS. Records of competence training are kept and maintained. Additional training is delivered to address gaps in competence (e.g., for staff who change roles). Guidance for all levels This is applicable only to staff with SMS obligations. Guidance for the Assured Level An evaluation of the effectiveness of the SMS training is not necessarily linked to the competence in a licensed role (e.g. ATCO, ATSEPs). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 46 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 17: Safety Communication Level A Level B Level C Level D Question 17.1 Informal Defined Managed Assured Arrangements Staff are informed about the safety and safety Staff have limited knowledge of SMS Relevant staff are informed when safety Safety is a key focus of internal The organisation regularly assesses the management standards relevant to their processes and procedures. actions or new safety management communications. effectiveness of its communication, and positions. procedures are introduced. addresses any deficiencies. Staff are informed when procedures have The organisation issues internal staff changed. communications that focus on safety and safety management. The organisation tailors its safety communications to meet the recipients' needs. Guidance for the Assured Level The organisation describes the process of how the assessment takes place and the corresponding reporting/correction process, including the date of the last review of the effectiveness of safety communication. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 47 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 17: Safety Communication Level A Level B Level C Level D Question 17.2 Informal Defined Managed Assured Arrangements Organisational-wide methods to record and Safety lessons learned are known only to The organisation intends to record and The organisation has a formal process for The organisation systematically shares all disseminate lessons learned and time-critical those who experience them. share lessons learned throughout the systematically sharing operational safety safety lessons learned throughout the safety information exist. organisation. lessons learned with appropriate staff. organisation at all appropriate levels. The organisation disseminates safety- The organisation regularly reviews its related information to all appropriate staff. lessons-learned dissemination process. Staff are given the appropriate means to react to communications and alert the organisation of any perceived problems. This is to be considered as outside of the regular occurrence reporting system. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 48 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for the Managed Level Examples of safety-related information are:  supplementary instructions;  temporary operating instructions;  safety notices. Guidance for the Assured Level In order to establish a track record, at least two reviews of the lessons-learned dissemination process are required to meet the requirement of this question. Alternatively, there should be a continuous monitoring process in place. In addition, the process should be formal in nature to justify this level. The regularity of the review should be agreed with the competent authority and be performed at least every 5 years. The results of the lessons- learned dissemination process should be used to drive improvement. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 49 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 17: Safety Communication Level A Level B Level C Level D Question 17.3 Informal Defined Managed Assured Arrangements Appropriate safety information and Safety data and information are treated as The organisation shares safety data and When required by regulation, the The organisation encourages the proactive knowledge is shared with industry confidential. There are no plans to information externally using informal organisation shares safety data and sharing of safety-related information with stakeholders. disseminate it to any industry stakeholders. processes. information nationally. other parties (including industry stakeholders) to drive safety improvement. Information disclosure complies with agreed When required by regulation, the publication and confidentiality policies / organisation shares safety data and The organisation actively shares safety data agreements. information with international bodies. with international bodies to drive safety improvement. The organisation has established a formal process to receive and act on safety data and information from external stakeholders. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 50 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for all levels Information disclosure should be consistent with the requirements of Regulation (EU) No 376/2014. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 51 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 5: Safety Promotion Study Area 17: Safety Communication Level A Level B Level C Level D Question 17.4 Informal Defined Managed Assured Arrangements A general public knowledgeable of the ANSP’s Safety-related performance information is The organisation makes safety-related The organisation makes high-level safety- The organisation makes safety performance performance through routine publication of not made available to the public under any performance information available to related performance information available information available to the general public achieved safety levels and trends. circumstances. selected authorities. according to regulatory requirements. beyond what is required by regulation. Guidance for all levels Information disclosure should be consistent with the requirements of Regulation (EU) No 376/2014. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 52 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Component 6: Interdependencies, Resilient system performance, buffers and trade-offs Study Area 18 Managing the interdependencies of complex operational environments and competitive business models Level A Level B Level C Level D Question 18.1 Informal Defined Managed Assured Arrangements Mature ANSPs sustain safe provision of Organisational business planning and Organisational business planning and The financial and personnel resources that are The organisation integrates safety fully into services through managing the organisation in strategy makes no formal allowance for safe strategy formally takes account of all safety needed to support safe production through business planning making provision of safe a way that recognises that system safety is at provision of service. regulatory requirements. safety promotion, safety improvement, production, in a traceable way, accessible risk from commercial and business models safety assurance and safety risk and subject to organisational governance. and targets. Such organisations embed safety Safety benefits are not systematically The safety consequences of business management are reviewed annually. Safety activities are resourced as a normal in organisational processes included in long-term investment decisions, strategies that emphasise efficiency at the Business plans are adjusted annually to business activity. although this may occur on an ad hoc basis. expense of the ability to adapt or limits ensure that these needs are met. The ANSP assigns and distributes resources, sources of resilience are not considered. Long term investment planning embeds both in terms of finances and personnel, to Safety benefits are not systematically Resource allocation for safe provision of provision of safety activities as a strategic support safe provision of services through included in changes to the functional system Safety is an independent organisation within services is assimilated into corporate corporate proposition. safety promotion, safety improvement, (including airspace design changes) other the organisation as a whole. business planning for operational and safety assurance and safety risk than on an ad-hoc basis. selected non-operational departments. The assessment of business models and/or It is acknowledged that business decisions business strategies on the dynamics and management. The emphasis in business planning is on cost- can influence safe provision of services. Trade-offs and sacrifices in operational capability of the organisation to deliver safe efficient service provision. decision making involve managing resource production takes into account consideration shortfalls with reduced resources within the of the buffers that are used in operational work system to draw upon to escalate and trade-offs and sacrifices in decision making. manage anomaly response. It strives to optimise operational agility and adaptive capacity. Financial and personnel resources are provided to enable the release of staff for Operational trade-offs and sacrifices in safety activities, such as training. decision making are modelled for effects and consequences. This leads to the identification and management of eroded buffers and sources of resilience. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 53 of 54 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(C) Appendix to the Annex: Questionnaire and associated verification guidance for the EoSM SKPI Guidance for all levels The financial and personnel resources that are needed to support safe production through safety promotion, safety improvement, safety assurance and safety risk management are reviewed annually. Business plans are adjusted annually to ensure that these needs are met. The financial calculations should include capital expenditure, staff costs (including transcription and support staff) that is budgeted for, allocated, and spent on:  organisational safety formally as part of the safety organisation and in direct provision of the compliance activities;  safety activities outside the formal provision of safety in the organisation, e.g. safety-promotion and improvement activities that are beyond the scope of the compliance requirements and is prospective and forward-thinking in its nature and outside compliance requirements;  its investment programme that is directly in support of the safety improvement and safety promotion, and that is prospective and forward-thinking. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 54 of 54 An agency of the European Union European Union Aviation Safety Agency Notice of Proposed Amendment 2019-10(B) Measurement of the safety key performance indicator and safety performance indicators in the SES Performance and Charging Scheme Material for the implementation and measurement of the safety key performance indicator (SKPI) and safety performance indicators (SPIs) for the Third Reference Period (RP3) of the SES ATM Performance and Charging Scheme (Commission Implementing Regulation (EU) 2019/317) RMT.0723 EXECUTIVE SUMMARY The objective of this Notice of Proposed Amendment (NPA) is to provide technical material regarding the implementation and measurement of the SKPI at the level of air navigation service providers (ANSPs) and the SPIs at both the State and ANSP level. The NPA and the indicators referred to above are linked to Commission Implementing Regulation (EU) 2019/317 laying down a performance and charging scheme in the single European sky and repealing Implementing Regulations (EU) Nos 390/2013 and 391/2013, being the Third Reference Period (RP3) of the SES Performance and Charging Scheme. This NPA proposes AMC and GM appropriate to the requirements of the SKPI of RP3 of the SES Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) 2019/317. The proposed amendments are expected to decrease the safety-reporting burden and reduce regulatory burden when compared with the AMC and GM for RP2 of the SES Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) No 390/2013. Further, the amendments are expected to facilitate stakeholders in complying with the safety performance requirements of the above-mentioned Commission Implementing Regulation. The SKPI reporting is restricted to ANSPs and, wherever possible, the SPIs will be calculated using occurrence data that has been reported to the European Central Repository under Regulation (EU) No 376/2014. Once consulted upon, the final material will be provided to the European Commission, whereupon it will be incorporated into a Commission Notice in relation to the SES ATM Performance and Charging Scheme Regulation. As such, it will not be published as AMC and GM through an EASA ED Decision nor as an Opinion. Please note that the material is presented in three sub-NPAs: — NPA 2019-10(A): the Explanatory Note; — NPA 2019-10(B): the Annex to the Explanatory Note, containing the material that is proposed to be adopted into a Commission Notice, which further describes the SKPI and SPIs, as defined in Commission Implementing Regulation (EU) 2019/317; — NPA 2019-10(C): the Appendix to the Annex, providing the questionnaire and associated verification guidance for the Effectiveness of Safety Management (EoSM) SKPI. This will also be adopted into the Commission Notice. Action area: Safety; systemic enablers; safety management Related rules: Commission Implementing Regulation (EU) 2019/317 Affected stakeholders: ANSPs; Member States (MSs) Driver: Efficiency/proportionality Rulemaking group: Yes Impact assessment: None Rulemaking Procedure: Standard 1.5.2018 19.9.2019 2019/Q4 2019/Q4 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 1 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Draft acceptable means of compliance (AMC) and guidance material (GM) for the implementation and measurement of the safety key performance indicator (SKPI) and other safety performance indicators (SPIs) supporting Commission Implementing Regulation (EU) 2019/317 (the SES performance and charging Regulation) TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 2 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Contents I General .................................................................................................................................................. 4 GM1 SKPI — General .......................................................................................................................... 4 II Effectiveness of the safety management KPI at ANSP level ................................................................ 7 AMC1 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — General................................................................................................................................................ 7 GM2 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — General................................................................................................................................................ 7 AMC2 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level ........ 9 Figure 1: Representation of the verification mechanism ............................................................. 12 GM3 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — Interdependencies ............................................................................................................................ 12 GM4 SKPI Measurement of the effectiveness of safety management (EoSM) KPI — ANSP level — Verification mechanism .................................................................................................................... 12 III Safety performance indicators (SPIs) ................................................................................................ 14 AMC3 Safety performance indicators (SPIs) for the monitoring of separation minima infringements (SMIs) and runway incursions (RIs) ........................................................................... 14 GM5 Safety performance indicators (SPIs) for the monitoring of separation minima infringements (SMIs) and runway incursions (RIs) ................................................................................................... 16 AMC4 Safety performance indicator (SPI) on automated safety data recording systems ............. 21 GM6 Safety performance indicator (SPI) on automated safety data recording systems ............... 22 AMC5 Safety performance indicator (SPI) for monitoring ATFM over-deliveries .......................... 24 GM7 Safety performance indicator (SPI) for monitoring ATFM over-deliveries ............................ 25 Figure 2: Example of hourly slices of 20-minute intervals of a regulated sector ......................... 26 Figure 3: Example of over-deliveries in 20-minute intervals in a regulated sector ...................... 26 Appendices............................................................................................................................................ 28 Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI — Questionnaire for the measurement of the effectiveness of safety management (EoSM) of ATS providers and associated guidance for verification by NSA/competent authority......................................................................................... 30 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 3 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ I General GM1 SKPI — General A. Purpose This Annex contains the acceptable means of compliance (AMC) and guidance material (GM) for measuring the safety key performance indicator (SKPI) and safety performance indicators (SPIs) in accordance with Commission Implementing Regulation (EU) 2019/3171 (the performance and charging scheme Regulation) for the Third Reference Period (RP3). AMC are non-binding standards adopted by the European Union Aviation Safety Agency (EASA) (hereinafter referred to as ‘EASA’) to illustrate means to establish compliance with the performance and charging scheme Regulation. When these AMC are complied with, the requirements on the measurement of the SKPI and SPIs in the performance scheme Regulation are considered as met. Should a Member State or an air navigation service provider (ANSP) wish to use different means to measure the SKPI or SPIs, they should: — inform EASA thereof, and — be able to demonstrate, by means of evidence, that the outcome of the application of any alternative means maintains the level of compliance with the performance and charging scheme Regulation and reaches a result that is comparable with the result of using the AMC in this Annex. B. Objective The objective of this Annex is to establish the method for the measurement and verification of the SKPI and SPIs under the performance scheme Regulation: (a) Effectiveness of safety management (EoSM) by ANSPs, which should be measured through a periodic answering of the questionnaires whose content is provided in the Appendix to AMC2 SKPI, GM3 SKPI and GM4 SKPI. The questionnaires, as completed by the ANSP subject to evaluation, and distributed in accordance with the performance and charging scheme Regulation, should be verified as detailed in AMC2 SKPI and GM4 SKPI; (b) Monitoring of separation minima infringement and runway incursion occurrence rates, which should be measured as detailed in AMC3 SPI and GM5 SPI; (c) Monitoring of the use of automatic safety data recording systems for monitoring and recording of separation minima infringements and runway incursions by the ANSPs, which should be measured as detailed in AMC4 SPI and GM6 SPI; and (d) Monitoring of the air traffic flow management (ATFM) over-deliveries, which should be measured as detailed in AMC5 SPI and GM7 SPI. 1 Commission Implementing Regulation (EU) 2019/317 of 11 February 2019 laying down a performance and charging scheme in the single European sky and repealing Implementing Regulations (EU) No 390/2013 and (EU) No 391/2013 (OJ L 56, 25.2.2019, p. 1) (https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1566812543741&uri=CELEX:32019R0317). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 4 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ C. Definitions and acronyms Definitions ‘Best (good) practice’ is a method, initiative, process, approach, technique or activity that is believed to be more effective at delivering a particular outcome than other means. It implies accumulating and applying knowledge about what is working and what is not working, including lessons learned and the continuing process of learning, feedback, reflection and analysis. ‘Risk’ refers to safety risk and means the combination of the overall probability or frequency of occurrence of a harmful effect induced by a hazard and the severity of that effect. ‘Safety culture’ means the shared beliefs, assumptions and values of an organisation and is part of the organisational culture. Acronyms ACC area control centre AMC acceptable means of compliance ANS air navigation service ANSP air navigation service provider APP approach control unit ATC air traffic control ATCO air traffic control officer ATFM air traffic flow management ATM air traffic management ATS air traffic services ECCAIRS European Coordination Centre for Accident and Incident Reporting Systems ECR European Central Repository EoSM effectiveness of safety management ERCS European Risk Classification Scheme FAB functional airspace block IFR instrument flight rules TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 5 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ GM guidance material KPI key performance indicator MO management objective MS Member State MTCD medium-term conflict detection NSA national supervisory authority NSA Coordination Platform NCP PRB Performance Review Body RAT risk analysis tool RI runway incursion RP reference period RMZ radio mandatory zone SA study area SKPI safety key performance indicator SLA service level agreement SMI separation minima infringement SMS safety management system SPI safety performance indicator TMA terminal manoeuvring area TMZ transponder mandatory zone TWR tower control unit UAC upper area control centre VFR visual flight rules TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 6 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ II Effectiveness of the safety management KPI at ANSP level AMC1 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — General GENERAL DESCRIPTION The EoSM indicator should be measured by verified responses to questionnaires as contained in this Annex. For each question, the response should indicate the level of implementation, characterising the level of performance of the reporting organisation. EFFECTIVENESS LEVELS AND EFFECTIVENESS SCORE When answering the questions, one of the following levels of implementation, A to D, should be selected: Level A is ‘Informal arrangements’ Level B is ‘Defined’ Level C is ‘Managed’ Level D is ‘Assured’ The specific requirements to achieve each level, A to D, is indicated for every question, as contained in this Annex. An effectiveness level should be selected only if all the elements described in the questionnaire as described in the Appendix to AMC2 SKPI, GM3 SKPI and GM4 SKPI are fully observed by an ANSP. If an ANSP has identified elements in various adjacent effectiveness levels, then it should take a conservative approach and select the lower effectiveness level for which all elements are covered. Based on the responses, an overall effectiveness score should be derived from the effectiveness levels selected by the ANSP against each question as described in AMC2, Section B. GM2 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — General A study area (SA) has been derived and adapted for each of the elements of the safety management system (SMS) as described in ICAO Annex 19, and has been aligned as far as reasonably practicable with Commission Implementing Regulation (EU) 2017/3732. For each SA, a question (or a set of questions) has been derived and the levels of effectiveness have been described. The available levels of effectiveness, and their intended meaning, are as follows: 1. Level A — Informal arrangements: SMS processes and/or requirements have not been agreed at the organisation level; they are either not routinely undertaken or depend on the individual assigned to the task. 2 Commission Implementing Regulation (EU) 2017/373 of 1 March 2017 laying down common requirements for providers of air traffic management/air navigation services and other air traffic management network functions and their oversight, repealing Regulation (EC) No 482/2008, Implementing Regulations (EU) No 1034/2011, (EU) No 1035/2011 and (EU) 2016/1377 and amending Regulation (EU) No 677/2011 (OJ L 62, 8.3.2017, p. 1) (https://eur-lex.europa.eu/legal- content/EN/TXT/?qid=1566813407294&uri=CELEX:32017R0373) TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 7 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ 2. Level B — Defined: SMS processes and/or requirements are defined but not yet fully implemented, documented or consistently applied. 3. Level C — Managed: SMS processes and/or requirements are fully documented and consistently applied. 4. Level D — Assured: Evidence is available to provide confidence that SMS processes and/or requirements are being applied appropriately and are delivering positive, measurable results. The questionnaire has been elaborated using the CANSO Standard of Excellence (SoE) as the basis and adapting it to the needs of the performance and charging scheme Regulation. Modifications have been minimised, in order to deviate as little as possible from the CANSO SoE questionnaire. Nonetheless, some differences have been introduced. The main deviations with respect to the CANSO SoE are as follows: 1. Study areas 6, 8, 9, and 10 of the SoE have been removed, as there are no corresponding requirements in the SMS components required by Commission Implementing Regulation (EU) 2017/373; 2. Maturity level E (Optimised) is not used, because this level is intended to set international best practices. Achieving level E in every ANSP or across every study area is unrealistic, and therefore is not targeted; 3. Study area 18 has been added as an optional component to capture how the ANSP deals with safety interdependencies, and trade-offs, serving as a proxy of the system resilience of the organisation; 4. Study area 1, safety culture, has been completely redrafted to be fit for purpose. EASA and the Performance Review Body (PRB) will monitor the performance of ANSPs regarding this indicator based on the received answers and on the results of the verification process by national supervisory authorities (NSAs) as presented in Figure 1 in AMC2 SKPI, Section C. The questionnaire’s sole intent is to monitor the performance (effectiveness) of ANSPs regarding ATM/ANS safety management. In order to facilitate this process for stakeholders, the questionnaire will be made available via an online tool, which will allow respondents to complete and submit their responses to the questionnaires. ANSPs are expected to provide evidence-based answers to these questionnaires, and a dedicated ‘Justification and evidence’ field together with a verification field have been provided to facilitate the validation of the claimed level by the NSA. In line with the responsibilities inherent in the system, the NSA of each Member State is responsible for verifying the ANSP responses and for submitting those responses per the requirements given in Commission Implementing Regulation (EU) 2019/137. The response levels assessed in the completed EoSM questionnaires should be used with the sole purpose of generating recommendations and associated plans for the improvement of safety management. These response levels should not be used to generate findings in the context of standardisation or oversight inspections. In accordance with the standardisation principles at Member States, if during an oversight inspection a finding is raised by the NSA in relation to the ANSP responses to the EoSM questionnaire, corrective action by the ANSP is required. Further, where a finding identifies that any of the questions in the TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 8 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ EoSM questionnaire is scored higher than it should be, the score should be corrected and lowered to the appropriate level of implementation. The outcome of oversight is not designed to be used for corrections of the scores towards a higher level of implementation. AMC2 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level The answers to the questionnaire should be used to measure the level of effectiveness in achieving the management objectives defined in this AMC. For each question, ANSPs should provide their NSA/competent authority with information on the level of effectiveness (or level of implementation) and evidence to justify their answer as indicated below. The questionnaire, which should be filled in by the ANSPs, is detailed in the Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI. A. Components, study areas (SAs) According to Commission Implementing Regulation (EU) 2019/317, the indicator is stated as follows: ‘The minimum level of the effectiveness of safety management to be achieved by air navigation service providers certified to provide air traffic services. This KPI shall be measured by the level of implementation of the following safety management objectives:’ For the sake of coherence in describing the EoSM in this document and the components of the ICAO Safety Management Framework, these safety management objectives are hereinafter referred to in this AMC as ‘components’, and they are as follows: (a) safety policy and objectives; (b) safety risk management; (c) safety assurance; (d) safety promotion; (e) safety culture. Each component addresses a set of SAs as follows: — Component 1: Safety culture — SA: Development of a positive and proactive organisational culture — Component 2: Safety policy and objectives — SAs: Safety policy Safety accountabilities Coordination of emergency response plan Safety management system documentation — Component 3: Safety risk management TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 9 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ — SA: Risk management process — Component 4: Safety assurance — SAs: Safety reporting Safety surveys and audits Safety performance monitoring Management of change Continual improvement of the SMS — Component 5: Safety promotion — SAs: Training and education Safety communication The SAs are further broken down into questions for which the ANSP respondents are expected to choose a level from the predetermined list of maturity levels that best describes the performance of the organisation with respect to the aim of that question. Organisations are reminded that in order to qualify for the chosen maturity level, all requirements as listed in the question must be met. The maturity level of an SA should be assigned considering the minimum maturity level achieved among the questions in that SA. Similarly, the maturity level of a component should be assigned considering the minimum maturity level achieved among the SAs in that component. B. Scoring In order to be able to measure quantitatively the overall effectiveness of safety management of the ANSP, the answers to the questions should be quantified. The responses provided by the ANSP on their questionnaires are assigned a numerical value from 1 to 4, corresponding to levels A to D. Level E is not assessed and has no value assigned. Each question has the same weight over the final overall score. The numerical value of each question should be added from the questionnaire responses and the final overall EoSM score is calculated as a percentage of the maximum score value possible. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 10 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Maturity level Questions A B C D E SA1-Q1 1 2 3 4 n/a SA1-Q2 1 2 3 4 n/a SA1-Q3 1 2 3 4 n/a SA2-Q1 1 2 3 4 n/a SA2-Q2 1 2 3 4 n/a SA3-Q1 1 2 3 4 n/a SA3-Q2 1 2 3 4 n/a SA3-Q3 1 2 3 4 n/a SA4-Q1 1 2 3 4 n/a SA5-Q1 1 2 3 4 n/a SA5-Q2 1 2 3 4 n/a SA5-Q3 1 2 3 4 n/a SA7-Q1 1 2 3 4 n/a SA7-Q2 1 2 3 4 n/a SA7-Q3 1 2 3 4 n/a SA11-Q1 1 2 3 4 n/a SA12-Q1 1 2 3 4 n/a SA13-Q1 1 2 3 4 n/a SA13-Q2 1 2 3 4 n/a SA14-Q1 1 2 3 4 n/a SA15-Q1 1 2 3 4 n/a SA15-Q2 1 2 3 4 n/a SA16-Q1 1 2 3 4 n/a SA16-Q2 1 2 3 4 n/a SA17-Q1 1 2 3 4 n/a SA17-Q2 1 2 3 4 n/a SA17-Q3 1 2 3 4 n/a SA17-Q4 1 2 3 4 n/a Mathematically, the effectiveness score for an ANSP ‘j’ is calculated as follows: 𝑞 100 ∗ ∑𝑘=1 𝑟𝑘 𝑆𝑗 = 4 ∗ 28 Where: Sj is the effectiveness score of the ANSP rkj is the numeric value of the response of an ANSP to question k q is the number of questions for which responses were provided by the ANSP Q is the total number of questions in the EoSM questionnaire, i.e. 28 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 11 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ C. Mechanism for verification The verification of the ANSP questionnaires by the NSA/competent authority should take place before the questionnaires and their results are submitted to EASA. The verification mechanism is presented in Figure 1. ANSPs should assign a focal point for the purpose of the verification process. EASA Verified results Verified results Verified results NSA 1 NSA 2 … NSA n Results Results Results ANSP 1 ANSP 2 … ANSP n ANSP 1 ANSP 2 … ANSP n ANSP 1 ANSP 2 … ANSP n Figure 1: Representation of the verification mechanism The competent authority/NSA may allocate the detailed verification task to a qualified entity. GM3 SKPI Measurement of the effectiveness of safety management (EoSM) at ANSP level — Interdependencies The questionnaire has been supplemented with a new component that captures how the ANSP manages interdependencies and trade-offs between safety and other business objectives. The main question to address is how the organisation assigns and distributes resources to ensure safe provision of ATS. This component is not targeted. — Component 6: Interdependencies, resilient system performance, buffers and trade-offs — SA Managing the interdependencies of complex operational environments and competitive business models GM4 SKPI Measurement of the effectiveness of safety management (EoSM) KPI — ANSP level — Verification mechanism A. VERIFICATION OF THE ANSP EoSM BY THE NSA/COMPETENT AUTHORITY When verifying the EoSM questionnaires completed by an ANSP, the competent authority/NSA may organise bilateral interview sessions. In these interview sessions, the NSA coordinator may ask the ANSP focal point some additional questions and request some additional evidence in order to verify the correctness of the answers provided for the questionnaire. It is the responsibility of the ANSP to complete the ANSP-level effectiveness-of-safety-management (EoSM) questionnaire and for the NSAs to verify the evidence submitted. When answering the questions, one out of four (from A to D) levels of implementation is to be selected. The ANSPs will select the implementation level that best describes their organisation, and provide evidence and a justification in support of the level selected. In order to ensure consistent interpretation of the questions, Table A presents a set of generic principles that are applicable to each maturity level, throughout the questionnaire. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 12 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Table A: Generic principles for each implementation level Level A — Informal Level B — Defined Level C — Managed Level D — Assured arrangements SMS processes and/or SMS processes and/or SMS processes and/or Evidence is available requirements have requirements are requirements are fully to provide confidence not been agreed at defined but not yet documented and that SMS processes the organisation level; fully implemented, consistently applied. and/or requirements they are either not documented or are being applied routinely undertaken consistently applied. appropriately and are or depend on the delivering positive, individual assigned to measurable results. the task. In addition, examples of expected outcomes for each question that align with each implementation level, together with additional explanations, are provided at the end of each SA group, where necessary, in the Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI. Respondents are reminded that the answers should be conservative and ALL required elements have to be in place for a certain level. This includes the generic elements from Table A, as well as the particular elements suggested by the questionnaire in the Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI. Even if a certain level has only one or two elements still missing, then the lower level with all elements in place have to be selected. B. COORDINATION BETWEEN THE COMPETENT AUTHORITIES/ NSAs FOR THE VERIFICATION OF THE ANSPs The competent authorities/NSAs might need better coordination between them in the verification process in order to achieve consistent and comparable results at European level. One potential solution could be the extension of the terms of reference for the NSA Coordination Platform (NCP) in the field of harmonisation of the verification mechanism of the SKPI at ANSP level. Notwithstanding the above and notwithstanding the fact that NSAs may delegate the verification task to a qualified entity, the responsibility for verification of the SKPI measurement at ANSP level lies with the competent authority/NSA. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 13 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ III Safety performance indicators (SPIs) AMC3 Safety performance indicators (SPIs) for the monitoring of separation minima infringements (SMIs) and runway incursions (RIs) A. SAFETY IMPACT For the determination of the occurrences with ‘safety impact’ that are used for monitoring runway incursions (RIs) and separation minima infringements (SMIs), only a subset of the occurrences that may represent a risk to aviation safety should be selected. The indicators set out in point 1.2(a) and 1.2(b) of Section 2 of Annex I should include occurrences whose safety risk grade is red or amber in the European Risk Classification Scheme (ERCS) matrix. These are the indicators at Member State level. The indicators set out in point 1.2(c) and 1.2(d) of Section 2 of Annex I should include occurrences whose risk analysis tool (RAT) ground severity classification is A, B, or C. These are the indicators at airport or ANSP level. B. EXPOSURE DATA For the calculation of indicators, the Network Manager should provide to the European Commission controlled flight hours within the Member States’ boundaries and controlled flight hours by the ATS units. The ANSPs should provide to the European Commission IFR and VFR movements at airports. C. DATA REPORTING AND DATA SOURCE For the calculation of the indicators related to SMIs and RIs within the scope of Commission Implementing Regulation (EU) 2019/317, Member States should provide the occurrence data making use of the existing safety occurrence data reporting mechanism under Regulation (EU) No 376/20143 and submitted to the European Central Repository (ECR). ANSPs and NSAs should ensure that the information provided through occurrence reporting under Regulation (EU) No 376/2014 contains the information needed to compute the performance indicators for monitoring SMIs and RIs. In particular, they should ensure that the following information is coded and reported: — For monitoring SMIs: — unambiguously identify the safety occurrences that are SMIs; — when the SMI occurred at the arrival or departure at an airport, the location indicator of the airport where it took place; — The ATS unit name, airspace type, class and FIR/UIR name; — information on whether, in the judgement of the investigators of the occurrence, the ATS or CNS contributed to the SMI, either directly or indirectly or none, as appropriate; 3 Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007 (OJ L 122, 24.4.2014, p. 18) (https://eur-lex.europa.eu/legal- content/EN/TXT/?qid=1566816757728&uri=CELEX:32014R0376). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 14 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ — RAT ground severity associated to the SMI, as obtained by the application of the RAT methodology by the ANSP; — ERCS risk grade associated to the SMI, as obtained by the application of the ERCS methodology by the State. — For monitoring RI: — unambiguously identify the safety occurrences that are RIs; — location indicator of the airport where the RI took place; — the ATS unit name, airspace type, class and FIR/UIR name; — information on whether, in the judgement of the investigators of the occurrence, the ATS or CNS contributed to the RI, either directly or indirectly or none, as appropriate; — RAT ground severity associated to the RI, as obtained by the application of the RAT methodology by the ANSP; and — ERCS risk grade associated to the RI, as obtained by the application of ERCS methodology by the State. When receiving from EASA an analysis report of the reported occurrence data measuring these performance indicators for the preceding year (January–December), the NSAs should: — validate the numbers presented in the report and advise of any identified discrepancies, together with supporting evidence; — respond to all the observations in the report; and — send a confirmation of the numbers presented and responses to the observations to EASA by the end of May each year. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 15 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ GM5 Safety performance indicators (SPIs) for the monitoring of separation minima infringements (SMIs) and runway incursions (RIs) The purpose of this GM is to explain the safety performance indicators, the data requirements and the process by which the number of SMIs and RIs will be measured. A. RUNWAY INCURSION (RI) The definition of RI is provided in Article 2(19) of Commission Implementing Regulation (EU) 2019/317, which is the same definition as that adopted by ICAO. It is repeated here for ease of reference: ‘“runway incursion” means any occurrence at an aerodrome involving the incorrect presence of an aircraft, vehicle or person on the protected area of a surface designated for the landing and take-off of aircraft;’ In order to determine whether an event is a runway incursion or not, the following explanation is provided: — the ‘incorrect presence’ is defined as the unsafe, unauthorised or undesirable presence, or movement of an aircraft, vehicle, or pedestrian, irrespective of the main contributor (e.g. ATC, pilot, driver, technical system). The ‘protected area of a surface designated for the landing and take-off of aircraft’ is defined as a minimum the physical surface of the runway and the strip distance out to the holding point appropriate to the visibility conditions at the time of the event. The RIs included in the indicator are those that occur at the airports included by the Member States in their performance plans, where the airports to be considered are specified. Article 1(3) of Commission Implementing Regulation (EU) 2019/317 establishes the minimum list of airports as those airports in the territory of the Member State with 80 000 IFR movements or more. Additional airports may be included in the performance plans according to Article 1(4) of the same Regulation. B. SEPARATION MINIMA INFRINGEMENT (SMI) SMI is defined in Article 2(20) of Commission Implementing Regulation (EU) 2019/317, which is in line with industry practices. It is repeated here for ease of reference: ‘“separation minima infringement” means a situation in which prescribed separation minima were not maintained between aircraft;’ It is important to note that both horizontal and vertical separation needs to be lost to trigger an SMI. It is understood that the infringement of the separation standard is between aircraft that are flying under the ATC services of the responsible ANSP. The SMI-related indicator covers aircraft in en-route, terminal and airport control zones. When the infringement occurs around an airport, only those occurrences attributed to the terminal navigation services around airports that are included in the Member States’ performance plans are included. Article 1(3) of Commission Implementing Regulation (EU) 2019/317 establishes the minimum list of airports as those airports in the territory of the Member State with 80 000 IFR movements or more. Additional airports may be included in the performance plans according to Article 1(4) of the same Regulation. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 16 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ C. LOCAL LEVEL versus UNION-WIDE LEVEL SPIs for the monitoring of SMIs and RIs at local level are established in Annex I, Section 2, point 1.2, paragraphs (a), (b), (c) and (d) of Commission Implementing Regulation (EU) 2019/317. They include indicators at Member State, ANSP and airport level. They are reproduced here for ease of reference: ‘(a) The rate of runway incursions at airports located in a Member State, calculated as the total number of runway incursions with a safety impact that occurred at those airports divided by the total number of IFR and VFR movements at those airports.’ The indicator set out in paragraph (a) is aggregated at airport level. It includes all RIs that have been reported under Regulation (EU) No 376/2014, independently of the main contributor, i.e. individuals, air operators, aerodromes, or ANSPs. As such, this indicator aims to capture trends in RIs at Member State level. ‘(c) The rate of runway incursions at an airport calculated as the total number of runway incursions with any contribution from air traffic services or CNS services with a safety impact that occurred at that airport divided by the total number of IFR and VFR movements at that airport.’ The indicator set out in paragraph (c) is aggregated at airport level. It includes only a subset of RIs that have been reported under Regulation (EU) No 376/2014 for which the ANSP was identified as having a contribution, either direct or indirect. This indicator aims to capture trends in RIs under the influence of the provider of ATC at the airport concerned. ‘(b) The rate of separation minima infringements within the airspace of all controlling air traffic services units in a Member State, calculated as the total number of separation minima infringements with a safety impact that occurred in that airspace divided by the total number of controlled flight hours within that airspace.’ The indicator set out in paragraph (b) is aggregated at Member State level. It includes all SMIs that have been reported under Regulation (EU) No 376/2014, independently of the main contributor, i.e. air operators or ANSPs. This indicator captures all SMIs that occur within the geographical boundary of a Member State, irrespective of which ANSP is providing the ATC service. ‘(d) The rate of separation minima infringements within the airspace where the air navigation service provider provides air traffic services, calculated as the total number of separation minima infringements with any contribution from air traffic services, or CNS services with a safety impact divided by the total number of controlled flight hours within that airspace.’ The indicator set out in paragraph (d) is aggregated at ANSP level. It includes only a subset of SMIs that have been reported under Regulation (EU) No 376/2014, for which the ANSP was identified as having a contribution, either direct or indirect. This indicator captures all SMIs that occur in the area where an ANSP provides its ATC services. SPIs for the monitoring of SMIs and RIs at Union level are established in Annex I, Section 1, point 1.2, paragraphs (a) and (b). These indicators are determined as the local-level indicators defined in point 1.2(a) and 1.2(b) of Section 2 and differ from them only in the level of aggregation. D. SAFETY IMPACT It is anticipated that Member States will classify occurrences in terms of safety risk according to the common European Risk Classification Scheme (ERCS) that the European Commission intends to adopt by means of implementing acts, as prescribed in Article 7 of Regulation (EU) No 376/2014. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 17 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ ERCS considers four levels of risk associated to occurrences, namely: ‘not safety related’, ‘low’, ‘medium’, or ‘high’. Each level is coloured in the ERCS risk matrix: green for ‘not safety related’ and ‘low’ risk occurrences, amber for ‘medium’ risk occurrences, and red for ‘high’ risk occurrences. The occurrences with safety impact considered in the computation of indicators for monitoring RIs and SMIs at Member State level refer to those that have been classified as ‘medium’ (amber) and ‘high’ (red) ERCS risk grade of the ERCS matrix. It is anticipated that ANSPs will classify occurrences in terms of severity according to the RAT methodology. This methodology classifies the severity of occurrences into five categories: ‘serious incident’ (A), ‘major incident’ (B), ‘significant’ (C), ‘not determined’ (D), and ‘no safety effect’ (E). The occurrences with safety impact considered in the computation of indicators for monitoring RIs and SMIs at ANSP level refer to those classified as ATM ground severity A, B, and C. The application of severity classification using the RAT methodology was formally introduced within the ATM performance scheme Regulations for RPs 1 and 2. At the end of RP2, the target for the application of severity classification using the RAT methodology by ANSPs was set to 100 % application for all reported SMIs and RIs with ATM ground severity A, B, and C, and it is anticipated that ANSPs will continue to apply it to these occurrences. In order to calculate the correct score and perform a proper analysis of the occurrence, it is good practice to determine both the ATM overall and ATM ground scores. E. ATS/CNS CONTRIBUTION There are two indicators, set up in points 1.2(c) and 1.2(d) of Section 2, that consider only those occurrences where, during the occurrence investigation, the ATS or CNS services contributed to the occurrence. This contribution is considered to be any causal or aggravating factor from the ATS or CNS ground services to a situation, in the air or on the ground, where an aircraft/vehicle/person has lost the required safety margins. In contrast, cases where there is no ‘ATS or CNS services contribution’ are: when the investigation shows evidence that there was no kind of causation/contribution/aggravation from the ATS or CNS ground services; and there was at no point in time any chance for the ATS or CNS ground services to detect and resolve a sudden/potential conflict in advance of a loss of required safety margins. F. EXPOSURE DATA The indicators for monitoring SMIs and RIs are normalised using the following exposure data: For RIs, the number of IFR and VFR movements at the airport is calculated with the sum of take-offs and landings performed under both IFR and VFR at that airport. Complete exposure data cannot be obtained from the Network Manager, which includes mainly IFR movements but a small portion of VFR flights. The Network Manager figures need to be complemented by the VFR traffic from the ANSP’s tower and airports. For SMIs, the number of controlled flight hours is measured as hours of flight under IFR that are under the separation control of ANSPs. The Network Manager is best placed to consistently report flight hours of ANSPs across Europe. As some ANSPs provide cross-border services, the measure of flight hours is based on two different measurements depending on the indicator. The indicator in paragraph (b) of Section 1 of Commission Implementing Regulation (EU) No 2019/317 is calculated using flight hours within the Member States’ boundaries, while the indicator in paragraph (d) of Section 1 of the same Regulation is calculated using flight hours controlled by a given ANSP. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 18 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ G. CODING PRACTICE IN ECCAIRS AND DATA COLLECTION PROCESS All ATM-related safety occurrences are required to be reported to the European Central Repository (ECR) under Regulation (EU) No 376/2014. It is anticipated that the common and specific mandatory data fields applicable to the occurrence will have been completed, in accordance with Annex I to that Regulation. As a minimum, the specific mandatory data fields should include those for aircraft-, air navigation services- and aerodrome-related occurrences. For the purposes of reporting under the performance scheme Regulation and for the facilitation of the computation of performance monitoring indicators, the following fields need to be coded for each occurrence record to provide the necessary information to allow proper computation of the indicators. The below fields are intended to be used for data extraction from the ECR and computation of the monitoring indicators for monitoring SMIs and RIs. Within ECCAIRS 5, the following additional fields need to be completed, as appropriate: Attribute ID Description Possible values Remarks 1049 Applicability SES Yes/No/Unknown This attribute provides an immediate performance scheme indication that the occurrence falls within the scope of the performance scheme, and will facilitate data extraction. Failing to code it will require airport information to discriminate whether the occurrence falls within the scope of the performance scheme. 5 Location indicator A four-letter code This attribute identifies the airport where group formulated in the occurrence took place. It is a accordance with the mandatory data field for RIs. For SMIs, it rules prescribed by is also needed as it may serve as filter to ICAO detect whether the occurrence falls within the scope of the performance scheme. 1109 ERCS risk grade Low (green), medium This attribute provides information about (amber) , high (red) the risk of the occurrence. It is used to identify those occurrences with safety impact at Member State level. 1095 ERCS score Row/column of the This attribute provides information about ERCS risk matrix the risk of the occurrence. It is used to identify those occurrences with safety impact at Member State level. 1074 Ground severity A, B, C, E, D, N This attribute provides information about the severity of the occurrence. It is used to identify those occurrences with safety impact at ANSP level. 390 Event type Predefined type of This attribute provides information on event, i.e. the type of occurrence to compute SMIs consequential and RIs. It is a common mandatory data events, equipment, operational, TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 19 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ personnel, field. For the performance scheme, Level organisational or 4 should be provided as follows: unknown For identifying RIs, the following event should be coded: Operational Aircraft flight operations Incursions Runway incursion by a person; Runway incursion by a vehicle/ equipment; or Runway incursion by an aircraft For identifying SMIs, the following event should be coded: Operational Aircraft flight operations Airborne conflict Separation minima infringement EASA will retrieve the data available in the ECR in order to calculate preliminary figures for the SPIs for monitoring SMIs and RIs. Member States will receive an analysis report sent by EASA based on the data submitted and coded by them, containing the number of applicable occurrences in their territory in the previous year (January–December). Observations related to the data extraction may be included. Member States will review this analysis report, confirm the occurrence numbers presented in the report, and respond to the observations. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 20 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ AMC4 Safety performance indicator (SPI) on automated safety data recording systems ANSPs should report to their competent authorities at the beginning of the application period and subsequently on an annual basis, the use of automatic safety data recording systems for the monitoring and recording of SMIs and RIs. Where automated safety data recording systems have been implemented, ANSPs should also answer the following questions: (a) What safety data is captured by the automated safety data recording systems? (b) How is the data captured used in support of the safety risk management framework? (c) How are just-culture organisation principles applied in gathering and using the safety data recorded? (d) How is the monitoring of data sources organised and how is it ensured that available data sources are utilised in a coherent way? (e) How is the data combined to provide the explanatory power to understand the context that led to safety occurrences and anticipate emerging risks? (f) How is the information from safety data analyses fed forward to risk assessment processes and to designers of future systems? (g) How is the information disseminated inside and outside the organisation? (h) Have obstacles of a technical, operational or cultural nature been identified that prevented the realisation of the full potential of a data-driven safety decision-making process? What are the main issues when using automated safety data recording systems? TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 21 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ GM6 Safety performance indicator (SPI) on automated safety data recording systems A. General The performance indicator on automated safety data recording systems (where implemented) in point 1.2(e) in Section 2 of Annex I is defined as: ‘[…], the use of these systems by the air navigation service providers, as a component of their safety risk management framework, for the purposes of gathering, storing and near-real time analyses of data related to, as a minimum, separation minima infringements and runway incursions.’ Beyond a narrow interpretation of the indicator as supporting a pure binary assessment of the performance, the indicator should be understood as an initiative to foster a proactive approach to safety management, one looking closely at day-to-day performance and including measures other than occurrences to anticipate risk. This is in line with Recommendation 7.1/1 — Data-driven decision- making from the thirteenth ICAO Air Navigation Conference (ANC) to facilitate ‘[…] data-driven decision-making in support of safety intelligence to support safety risk management’. This guidance material aims to assist Member States, NSAs and ANSPs in using automated safety data recording systems in the implementation of data-driven safety decision-making processes. The monitoring of this indicator during RP3 will provide key information to the forthcoming development of standardised risk-based decision-making policies and best practices for the design and parameterisation of safety-monitoring tools and models. B. Digitalisation and moving towards an early-warning capability for ATM Together with the massive amount of safety-related information that aviation generates today, as well as the increasingly rare accidents and serious incidents from which to learn and mitigate, goes the potential for a fundamental change in the mindset towards a more proactive, meaning-anticipative, collaborative, meaning-sharing, and performance-based approach to safety management. With the impending rise of information technology and overall digitalisation and rising automation of ATM, the pace of data creation can only increase. Obviously, data mining does not replace the technical and operational competencies of the ATM community and while it reduces uncertainty, it does not eliminate it, but it contributes to create safety intelligence. In particular, data helps in identifying and investigating the weak signals that could eventually result in catastrophic events. Therefore, today, the usage of automated safety data recording systems paves the way towards an early-warning capability for ATM with the aim to: — detect unsafe trends and implement changes that remove these threats before a serious event or worse happens; — react within a particular timescale that depends on the rate of trend progression; — not raise ‘false alarms’, nor lead to disproportionate focus on low-priority issues, or lead to unanticipated side effects; and — reach all those needed to ensure an aviation-system-wide reaction if the problem is generic, or localised reaction if it is a localised issue. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 22 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ C. Functional model The sequence of steps or functions (building upon automated safety data recording systems) that are needed for an early-warning function for ATM are as follows: — monitoring of data sources in ATM in a systematic and coherent way, in particular with respect to the specification of surrogates for accidents and incidents and setting of triggers for identifying adverse events and signals; — filtering, i.e. determining what is a ‘signal’ and what is ‘noise’, using statistical and risk-based criteria for deciding when to further analyse a potential trend or key occurrence; — trend identification to determine the exact nature of the safety issue; — getting sufficient understanding to estimate the risk priority and to prepare for mitigation measures. This should ensure that disproportionate focus does not occur, and that undesirable side effects are not generated. ‘Deconstructing’ the data should rely on a technical-/operational-centred approach to ensure the right balance between a current issue and others that are pending; — developing mitigation measures to deal with the issue and prevent its recurrence and/or propagation; — disseminating and engaging, i.e. letting the right people know; — verifying and confirming that the problem has gone away building upon the never-ending stream of data while paying due attention to the potential ‘Hawthorne effect’, which means the attention paid to an issue may mean it disappears for a time, then resurfaces; — documenting thereby ensuring that the whole process for an identified issue has been recorded so that if it recurs or a similar problem arises, the safety ‘thinking’ and analysis is available for future users/analysts. Documentation at this level also allows deeper ‘learning’ to occur, e.g. across issues. A larger picture may emerge. It would also save time and resources if problems resurface or ‘mutate’ into related problems; and — feeding forward the information from analyses to the risk assessment processes and to designers of future systems. D. Fundamental components Four fundamental components in the usage of automated safety data recording systems in support of the ‘safety risk management’, ‘safety achievement’, ‘safety assurance’, and ‘safety promotion’ elements of the SMS are: 1. the involvement of data analysts, data scientists, predictive modellers, statisticians and other analytics professionals to structure and analyse growing volumes of data to uncover information including hidden patterns, unknown correlations, etc.; 2. the interactive visualisation of the structured safety data to support the safety, technical and operational analyses; 3. the involvement of safety, operational and technical expertise to comprehend the data and prioritise the actions needed to ensure safe ATM operation; and 4. the gathering of the safety data and information in a just-culture organisational environment. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 23 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ AMC5 Safety performance indicator (SPI) for monitoring ATFM over-deliveries At ATC sector level, the ATFM over-deliveries (OVD) safety performance indicator should be calculated as the ratio of 20-minute slices with over-delivery aircraft in the ATFM regulated sector versus the total number of 20-minunte slices during the ATFM regulated duration. To determine whether an hourly slice is over-delivered, the number of actual flight entries in the regulated sector (NB_FLT_ACTUAL) should be compared with the regulated flight rate for the same time interval (REG_RATE) that is imposed in the ATFM regulation. When the actual entries are above 110 % of the regulated rate, then the slice should be considered over-delivered. The definition should exclude the regulation with a zero rate (e.g. airspace closures) as it makes the comparison meaningless. The Network Manager (NM) should report to EASA at the beginning of the application period and subsequently, on an annual basis, the OVD SPI aggregated at each ACC and SES areas. The time interval to monitor is each entire year. To aggregate the OVD SPI for the combination of geographical area and yearly interval, the total number of slices with over-delivery are divided by the total number of slices for the regulations within the reporting scope. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 24 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ GM7 Safety performance indicator (SPI) for monitoring ATFM over-deliveries A. Definition The purpose of this guidance is to explain the ATFM OVD SPI, its calculation and how it will be monitored. The OVD SPI is defined in Annex I, Section 3, point 2.2 as: ‘The ATFM over-deliveries above the capacity limits of a sector declared by the air navigation service provider where ATFM regulations are imposed, calculated as follows: (a) the ratio between the time that the number of flights exceeds by more than 10% the capacity limits of a sector declared by the air navigation service provider where ATFM regulations are imposed, and the total time where ATFM regulations are imposed, calculated for the whole calendar year and for each year of the reference period; (b) for the purposes of this indicator, the regulated time is divided in overlapping hourly slices at every 20-minutes interval.’ An ATFM regulation is a traffic flow measure that aims to protect a node that may potentially be overloaded by limiting the maximum rate of aircraft entering the node. The ATFM regulation is requested by affected ANSPs whenever expected demand exceeds available capacity of the node and will affect a number of flights that enter the node in a time period. Flights entering a regulated sector during the regulation period are subject to that regulation and may be assigned ATFM slots by the Network Manager. An ATFM regulation is, therefore, characterised by a regulated time duration, by the acceptable rate of flights that enter the regulated sector, and the flights affected, also known as traffic volume (TV). The traffic volume is created based on either a sector in the airspace, in which case falls within the scope of the indicator definition, or a significant point in the airspace or an airport/group of airports, which falls/fall outside the scope. In order to determine the total regulated time for an ATFM regulation applied on a traffic volume, the method considers the interval between the regulation start time and regulation end. The regulation end is defined as: 1. either the last regulation end time indicated when a regulation is created and at subsequent extensions, if any, when the regulation is not cancelled; 2. or the time at which the regulation was cancelled, when this happens before the declared regulation end time. The regulated time is further divided in ‘overlapping hourly slices at every 20-minute intervals’. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 25 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ For example, a regulation starting at 10:40 and ending at 11:40 will have 7 overlapping hourly slices defined as illustrated in Figure 2: 10:40 11:00 11:20 11:40 12:00 12:20 12:40 13:00 13:20 13:30 Slice 1: 10:40-11:40 Slice 2: 11:00-12:00 Slice 3: 11:20-12:20 Slice 4: 11:40-12:40 Slice 5: 12:00-13:00 Slice 6: 12:20-13:20 Slice 7: 12:40-13:30 Figure 2: Example of hourly slices of 20-minute intervals of a regulated sector The last slice is the first hourly slice reaching the regulation end; in the example above it is the 12:40 to 13:30 (this is the only slice that can be less than 1 hour). B. Example of the OVD calculation indicator Figure 3 is an example of an ATFM regulation that was imposed over a certain sector with a regulated rate of 35 flights/hour. The ATFM regulation duration applied from 10:40 until 13:30. The orange bars in the graph depict 20-minute slices that were over-delivered, while the green bars depict 20-minute slices where the actual flight entries were below the regulated rate. In this example, the OVD indicator value is OVD = 3/7 = 42.9 %. Actual entries vs Regulated rate (35/hour) Slices in red are over-delivered 45 41 39 39 40 36 34 35 35 30 28 25 20 15 10 5 0 Slice 1: Slice 2: Slice 3: Slice 4: Slice 5: Slice 6: Slice 7: 10:40-11:40 11:00-12:00 11:20-12:20 11:40-12:40 12:00-13:00 12:20-13:20 12:40-13:30 Actual entries Regulated rate Figure 3: Example of over-deliveries in 20-minute intervals in a regulated sector The number of actual flight entries in the TV for each 20-minute slice is calculated from the current tactical flight model (CTFM) profile generated by the NM system (whenever a flight fulfils that the CTFM entry time ≥ slice start time and the CTFM entry time < slice end time). To this, a correction will be applied for the airspace un-anticipated flights4 that, although geographically are crossing the 4 Airspace un-anticipated traffic are flights that are not planned to enter the TV based on the last filed flight plan but that are actually entering the TV by deviating from flight plan. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 26 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ regulated sector, from an operational perspective are not under the control of that sector (non- operational un-anticipated traffic). The objective is to avoid ‘false positives’, i.e. situations when an over-delivery seems to have occurred while in reality there was none5. This categorisation will be implemented in the NM reporting system during RP3. The technical system of the NM generates and archives the data used for OVD monitoring: 1. regulated TVs and associated reference locations; 2. ATFM regulation start, end, and cancellation times; 3. regulated rates; 4. number of actual entries in the regulated TVs for each slice; the categorisation of non- operational un-anticipated traffic will be available during RP3. C. Level of aggregation of the OVD indicator The OVD indicator can be determined for any combination of geographical areas (TV, ACC, SES area, NM area) or time intervals (daily, monthly, yearly). To aggregate the OVD indicator for the combination of geographical area and time interval, the total number of slices with over-delivery are divided by the total number of slices for the regulations within the reporting scope. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 27 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Appendices The appendix below will appear as a separate document (sub-NPA 2019-10(C)) to this Annex: — Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI — Questionnaire for the measurement of the effectiveness of safety management (EoSM) of ATS providers and associated guidance for verification by the NSA/competent authority The Appendix contains the AMC of the outcomes expected in each EoSM question associated to the corresponding study area (SA) and component/management objective, together with the expected outcome of the fulfilment of the objectives of EoSM for each level of maturity implementation for each question. These AMC appear in the tables included in the Appendix coloured in the grey cells of those tables. In addition, each question included in the tables contains guidance material with additional explanations, when necessary, useful for the verification by the NSA/competent authority. These GM appear in the tables included in the Appendix coloured in the white cells of those tables. The following picture depicts the elements contained in each table: TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 28 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 29 of 30 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(B) Annex: Draft AMC and GM ‘SKPI & SPIs’ Appendix to AMC3 SKPI, GM3 SKPI and GM4 SKPI — Questionnaire for the measurement of the effectiveness of safety management (EoSM) of ATS providers and associated guidance for verification by NSA/competent authority Note: Please refer to sub-NPA 2019-10(C) TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 30 of 30 An agency of the European Union European Union Aviation Safety Agency Notice of Proposed Amendment 2019-10(A) Measurement of the safety key performance indicator and safety performance indicators in the SES Performance and Charging Scheme Material for the implementation and measurement of the safety key performance indicator (SKPI) and safety performance indicators (SPIs) for the Third Reference Period (RP3) of the SES Performance and Charging Scheme (Commission Implementing Regulation (EU) 2019/317) RMT.0723 EXECUTIVE SUMMARY The objective of this Notice of Proposed Amendment (NPA) is to provide technical material regarding the implementation and measurement of the SKPI at the level of air navigation service providers (ANSPs) and the SPIs at both the State and ANSP level. The NPA and the indicators referred to above are linked to Commission Implementing Regulation (EU) 2019/317 laying down a performance and charging scheme in the single European sky and repealing Implementing Regulations (EU) Nos 390/2013 and 391/2013, being the Third Reference Period (RP3) of the SES Performance and Charging Scheme. This NPA proposes AMC and GM appropriate to the requirements of the SKPI of RP3 of the SES Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) 2019/317. The proposed amendments are expected to decrease the safety-reporting burden and reduce regulatory burden when compared with the AMC and GM for RP2 of the SES Performance and Charging Scheme as provided for under Commission Implementing Regulation (EU) No 390/2013. Further, the amendments are expected to facilitate stakeholders in complying with the safety performance requirements of the above-mentioned Commission Implementing Regulation. The SKPI reporting is restricted to ANSPs and, wherever possible, the SPIs will be calculated using occurrence data that has been reported to the European Central Repository under Regulation (EU) No 376/2014. Once consulted upon, the final material will be provided to the European Commission, whereupon it will be incorporated into a Commission Notice in relation to the SES Performance and Charging Scheme Regulation. As such, it will not be published as AMC and GM through an EASA ED Decision nor as an Opinion. Please note that the material is presented in three sub-NPAs: — NPA 2019-10(A): the Explanatory Note; — NPA 2019-10(B): the Annex to the Explanatory Note, containing the material that is proposed to be adopted into a Commission Notice, which further describes the SKPI and SPIs, as defined in Commission Implementing Regulation (EU) 2019/317; — NPA 2019-10(C): the Appendix to the Annex, providing the questionnaire and associated verification guidance for the Effectiveness of Safety Management (EoSM) SKPI. This will also be adopted into the Commission Notice. Action area: Safety; systemic enablers; safety management Related rules: Commission Implementing Regulation (EU) 2019/317 Affected stakeholders: ANSPs; Member States (MSs) Driver: Efficiency/proportionality Rulemaking group: Yes Impact assessment: None Rulemaking Procedure: Standard 1.5.2018 19.9.2019 2019/Q4 2019/Q4 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 1 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) Table of contents Table of contents 1. About this NPA...................................................................................................................... 3 1.1. How this NPA was developed ................................................................................................... 3 1.2. How to comment on this NPA .................................................................................................. 4 1.3. The next steps .......................................................................................................................... 4 2. In summary — why and what ................................................................................................ 5 2.1. Why we need to change the rules — issue/rationale .............................................................. 5 2.2. What we want to achieve — objectives ................................................................................... 5 2.3. How we want to achieve it — overview of the proposals........................................................ 5 2.4. What are the expected benefits and drawbacks of the proposals .......................................... 6 3. Proposed amendments and rationale in detail ....................................................................... 8 4. Impact assessment (IA).......................................................................................................... 8 5. Proposed actions to support implementation ........................................................................ 8 6. References ............................................................................................................................ 9 6.1. Related regulations................................................................................................................... 9 6.2. Related decisions ...................................................................................................................... 9 6.3. Other reference documents ..................................................................................................... 9 7. Quality of the document...................................................................................................... 10 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 2 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 1. About this NPA 1. About this NPA 1.1. How this NPA was developed The purpose of this NPA is to provide technical material to support compliance with Commission Implementing Regulation (EU) 2019/3171 (the ‘performance scheme Regulation’) as applicable to RP3 of the SES Performance and Charging Scheme (2020–2024). The European Commission tasked EASA per Article 75(2)(h) of the EASA Basic Regulation (Regulation (EU) 2018/1139)2: ‘2. For the purposes of ensuring the proper functioning and development of civil aviation in the Union in accordance with the objectives set out in Article 1, the Agency shall: (h) contribute, for matters covered by this Regulation, upon request by the Commission, to the establishment, measurement, reporting and analysis of performance indicators, where Union law establishes performance schemes relating to civil aviation;’ and as per Article 93 ‘Implementation of Single European Sky’ ‘The Agency shall, where it has the relevant expertise and upon request, provide technical assistance to the Commission, in the implementation of the Single European Sky, in particular by: (b) contributing, in matters covered by this Regulation, in cooperation with the Performance Review Body provided for in Article 11 of Regulation (EC) No 549/2004, to the implementation of a performance scheme for air navigation services and network functions;’ The European Union Aviation Safety Agency (EASA) developed this NPA in line with the Basic Regulation and the Rulemaking Procedure3. This rulemaking activity is included in the latest European Plan for Aviation Safety (EPAS)4 under rulemaking task (RMT).0723. The text of this NPA has been developed by EASA based on the input of the RMT.0723 Rulemaking Group (RMG). Subsequent to this tasking, upon advice given to DG MOVE by the European Commission’s Legal Service, the Commission advised EASA not to issue an ED Decision. While the NPA should be publicly consulted and EASA shall per the Rulemaking Process produce a comment-response document (CRD), 1 Commission Implementing Regulation (EU) 2019/317 of 11 February 2019 laying down a performance and charging scheme in the single European sky and repealing Implementing Regulations (EU) No 390/2013 and (EU) No 391/2013 (OJ L 56, 25.2.2019, p. 1) (https://eur- lex.europa.eu/legal-content/EN/TXT/?qid=1566568320291&uri=CELEX:32019R0317). 2 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1) (https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1535612134845&uri=CELEX:32018R1139). 3 EASA is bound to follow a structured rulemaking process as required by Article 115(1) of Regulation (EU) 2018/1139. Such a process has been adopted by the EASA Management Board (MB) and is referred to as the ‘Rulemaking Procedure’. See MB Decision No 18-2015 of 15 December 2015 replacing Decision 01/2012 concerning the procedure to be applied by EASA for the issuing of opinions, certification specifications and guidance material (http://www.easa.europa.eu/the-agency/management-board/decisions/easa-mb-decision-18-2015- rulemaking-procedure). 4 https://www.easa.europa.eu/document-library/general-publications?publication_type%5B%5D=2467 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 3 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 1. About this NPA the finalised output shall be forwarded directly to DG MOVE for inclusion in a Commission Notice covering RP3 of the SES Performance and Charging Scheme. It is hereby submitted to all interested parties5 for consultation. 1.2. How to comment on this NPA Please submit your comments using the automated Comment-Response Tool (CRT) available at http://hub.easa.europa.eu/crt/6. The deadline for submission of comments is 22 October 2019. 1.3. The next steps Following the closing of the public commenting period, EASA will review all the comments received. The comments received on this NPA and the EASA responses to them will be reflected in a CRD. The proposed AMC and GM will be amended where necessary on the basis of this consultation. The final AMC and GM will be submitted to the European Commission, which will use it as a technical basis in order to prepare a Commission Notice supporting RP3 of the SES Performance and Charging Scheme. As such, EASA will not issue an Opinion following the conclusion of this NPA. However, it will publish the CRD and an Explanatory Note, reflecting on the inputs and amendments to the material proposed in this NPA. 5 In accordance with Article 115 of Regulation (EU) 2018/1139, and Articles 6(3) and 7 of the Rulemaking Procedure. 6 In case of technical problems, please contact the CRT webmaster ([email protected]). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 4 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 2. In summary — why and what 2. In summary — why and what 2.1. Why we need to change the rules — issue/rationale Commission Implementing Regulation (EU) No 390/20137 laying down a performance scheme for air navigation services and network functions (the performance scheme Regulation) is in the process of being recast in preparation for RP3 of the SES Performance and Charging Scheme, which shall commence in 2020 and end in 2024. As a result of the Commission letter of 20 March 2018, EASA is tasked with preparing the necessary acceptable means of compliance (AMC) and, where necessary, the appropriate guidance material (GM) for the implementation and measurement of the safety key performance indicator (SKPI) and other safety performance indicators (SPIs) that will be applicable to the Safety Key Performance Area during RP3 in accordance with the provisions of the recast Implementing Regulation (EU) 2019/317 on the SES Performance and Charging Scheme. Having in mind that technical input from EASA stakeholders were needed for the task, it was decided that a rulemaking group would be established for the purposes of this task. The AMC and GM proposed have been developed under RMT.0723 and are based upon the outputs of the RP3 Development Working Group, which in 2016 was led by EASA to create a proposal for the S(K)PIs to be used in RP3. 2.2. What we want to achieve — objectives — To maintain, update and develop, as necessary, the technical material for the implementation and measurement of the SKPI that will be applicable to the Safety Key Performance Area in RP3. — To maintain, update and develop, as necessary, the technical material for the implementation and measurement of the SPIs that will be applicable to the RP3 Safety Key Performance Area. 2.3. How we want to achieve it — overview of the proposals In 2016, at the request of the European Commission, EASA, supported by a working group of experts drawn from Member States’ national aviation authorites (NAAs) and industry, developed the RP3 SKPI WG Report, ‘Indicator Proposals for RP3’ for application in the Safety Key Performance Area (hereinafter referred to as the ‘Report’). The contents of this Report, together with inputs from other stakeholders, have been used by the European Commission in drafting the legislative proposal for RP3. The proposals in this NPA follow the conclusions of the Report, as far as they covered the S(K)PIs as required by the performance scheme Regulation. As a result, the Effectiveness of Safety Management SKPI, applied at ANSP level only, has been developed based on the CANSO Standard of Excellence (hereinafter referred to as ‘the SoE’) measurement tool. Although it has been adapted to meet the needs of the performance and charging scheme Regulation, modifications have been minimised, in order to deviate as little as possible from the CANSO SoE questionnaire. Nonetheless, some differences have been introduced. The main deviations with respect to the CANSO SoE are as follows: 7 Commission Implementing Regulation (EU) No 390/2013 of 3 May 2013 laying down a performance scheme for air navigation services and network functions (OJ L 128, 9.5.2013, p. 1) (https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1566570852688&uri=CELEX:32013R0390). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 5 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 2. In summary — why and what — Study areas 6, 8, 9, and 10 of the SoE have been removed, as there are no corresponding requirements in the SMS components required by Commission Implementing Regulation (EU) 2017/3738; — Maturity level E (Optimised) is not used, because this level is intended to set international best practices. Achieving level E in every ANSP or across every study area is unrealistic, and therefore is not targeted; — Study area 18 has been added as an optional component to capture how the ANSP deals with safety interdependencies, and trade-offs, serving as a proxy of the system resilience of the organisation; — Study area 1, safety culture, has been completely redrafted to be fit for purpose. In addition, the Report proposed the reduction of the number of SPIs. These proposed SPIs are separation minima infringements and runway incursions, and they shall be defined as rates normalised by the appropriate exposure data. They shall be defined in such way that their measurement will rely on the data that is collected under Regulation (EU) No 376/20149, which is stored in the European Central Repository. The technical material should define the processes to collect the information that is required to populate the SPIs. The SPIs, to be measured at Member State, aerodrome or ANSP level, are defined in the performance scheme Regulation. The technical material sets out the definitions of the data to be reported and the collection methods. The indicators have been designed to use the European Central Repository, established under Regulation (EU) No 376/2014, as the source of occurrence data. Data from the Network Manager and the ANSPs are proposed as the source of exposure data. 2.4. What are the expected benefits and drawbacks of the proposals SKPI — Effectiveness of safety management It is expected that by aligning with the CANSO SoE, the efficiency of the performance scheme will be greatly improved. In addition, the CANSO SoE has been updated and improved during RP2, thus the proposed means of measuring the effectiveness of safety management reflects more recent experience and modern safety management approaches. There is not complete alignment with the CANSO SoE, therefore some small additional work will be required for ANSPs in responding to questions on safety culture. SPIs — The rate of runway incursions and separation minima infringements with a safety impact In defining the European Central Repository as the data source, the reporting and calculation of these performance indicators involves minimal additional effort on the part of the ANSPs and national supervisory authorities (NSAs). All the occurrence reports required for the performance scheme Regulation are already reportable under Regulation (EU) No 376/2014. Some additional data fields are 8 Commission Implementing Regulation (EU) 2017/373 of 1 March 2017 laying down common requirements for providers of air traffic management/air navigation services and other air traffic management network functions and their oversight, repealing Regulation (EC) No 482/2008, Implementing Regulations (EU) No 1034/2011, (EU) No 1035/2011 and (EU) 2016/1377 and amending Regulation (EU) No 677/2011 (OJ L 62, 8.3.2017, p. 1) (https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1566571320779&uri=CELEX:32017R0373). 9 Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation, amending Regulation (EU) No 996/2010 of the European Parliament and of the Council and repealing Directive 2003/42/EC of the European Parliament and of the Council and Commission Regulations (EC) No 1321/2007 and (EC) No 1330/2007 (OJ L 122, 24.4.2014, p. 18) (https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1566573102633&uri=CELEX:32014R0376). TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 6 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 2. In summary — why and what needed to efficiently identify the occurrences applicable to the scheme; however, these have been minimised. In defining safety impact as those occurrences with a high severity RAT-Ground score (ANSP level) or a medium/high risk ERCS score (Member State level), the safety impact will be assessed using a methodology that is already or shortly to be in use by EASA stakeholders. ANSPs have been using the severity classification of the RAT methodology throughout RP2 and Member States will soon be required to risk-classify occurrences using the European Risk Classification Scheme (ERCS) as per Regulation (EU) No 376/2014. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 7 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 3. Proposed amendments and rationale in detail 4. Impact assessment (IA) 5. Proposed actions to support implementation 3. Proposed amendments and rationale in detail The proposed S(K)PIs are described in full in the Annex (please refer to sub-NPA 2019-10(B)). 4. Impact assessment (IA) No impact assessment has been conducted for this NPA. This is because the S(K)PIs are outlined in the performance scheme Regulation, which has been assessed and consulted on by the European Commission. 5. Proposed actions to support implementation Implementation support to ANSPs and Member States is proposed to be managed via the normal activities of the SES Performance and Charging Scheme. Additional support on coding and analysis will be provided to Member States via the Network of Analysts. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 8 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 6. References 6. References 6.1. Related regulations — Commission Implementing Regulation (EU) No 390/2013 of 3 May 2013 laying down a performance scheme for air navigation services and network functions (OJ L 128, 9.5.2013, p. 1) — Commission Implementing Regulation (EU) 2019/317 laying down a performance scheme for air navigation services and network functions (OJ L 56, 25.2.2019, p. 1) — Regulation (EU) No 376/2014 of the European Parliament and of the Council on the reporting, analysis and follow-up of occurrences in civil aviation (OJ L 122, 24.4.2014, p. 18) 6.2. Related decisions — ED Decision 2014/035/R of 16 December 2014 adopting Acceptable Means of Compliance and Guidance Material for point 1 of Section 2 of Annex I to Regulation (EU) No 390/2013 and repealing Decision 2011/017/R of the Executive Director of the Agency of 16 December 2011 — ‘AMC and GM for the implementation and measurement of safety (Key) Performance Indicators (S(K)PIs) — Issue 2’ — ED Decision 2015/028/R of 17 December 2015 amending acceptable means of compliance and guidance material for point 1 of section 2 of Annex I to Regulation (EU) No 390/2013 — ‘AMC/GM to SKPI — Issue 2, Amendment 1’ 6.3. Other reference documents — Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.08.2018, p. 1) — Regulation (EC) No 549/2004 of the European Parliament and of the Council of 10 March 2004 laying down the framework for the creation of the single European sky (the framework Regulation) (OJ L 096, 31.3.2004, p. 1) — CANSO Standard of Excellence in Safety Management Systems v2.1, Civil Air Navigation Services Organisation, 2015 — EASA RP3 SKPI WG Report ‘Indicator Proposals for RP3’, European Aviation Safety Agency, 2016 TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 9 of 10 An agency of the European Union  European Union Aviation Safety Agency NPA 2019-10(A) 7. Quality of the document 7. Quality of the document If you are not satisfied with the quality of this document, please indicate the areas which you believe could be improved and provide a short justification/explanation: — technical quality of the draft proposed rules and/or regulations and/or the draft proposed amendments to them — text clarity and readability — quality of the impact assessment (IA) — others (please specify) Note: Your replies and/or comments to this section shall be considered for internal quality assurance and management purposes only and will not be published in the related CRD. TE.RPRO.00034-009 © European Union Aviation Safety Agency. All rights reserved. ISO 9001 certified. Proprietary document. Copies are not controlled. Confirm revision status through the EASA intranet/internet. Page 10 of 10 An agency of the European Union