The NEXES Key Performance Indicators

The NEXES Action aims to assess the impact of the adoption of Internet-enabled technologies by emergency services throughout Europe and even beyond Europe. Emergency services are quite different, due to their location and organisation.
How to assess any impact given very different subjects?

NEXES is a Research and Innovation Action about NEXt generation Emergency Services. Emergency services comprise fire brigades, police organisations, ambulance services and possibly other search and rescue organisations: services provided by the government for the protection, safety and security of its citizens. Emergency services situated in a city have goals such as an ambulance must arrive at the incident location within 10 minutes of calling. Emergency services situated in the countryside may have different goals, perhaps an arrival time of 20 minutes is required. Each country, within Europe and beyond, has a similar, yet different culture and organisation of its emergency services.

Emergency services that adopt new communication capabilities change. Part of the puzzle is being able to compare a ‘previous’ and ‘current’ emergency service. This difference within emergency services are a challenge for NEXES. NEXES strives to provide a set of key performance indicators – that is, evaluations of the performance – of emergency services that are about the results of adopting Internet-enabled technologies. These key performance indicators must assess ‘the impact’ of using the new technological possibilities.

And that’s where the challenge lies. How to assess the impact of the new technological possibilities, if the ‘subjects’ are similar, yet different? How to assess ‘before’ and ‘after’ the adoption of new technologies, and evaluate the gains? And how can key performance indicators help?

Key Performance Indicators (KPIs) are a well-known instrument to assess organisational performance. NEXES investigates the use of KPIs to assess the impact of its beyond state-of-the-art solutions for emergency services in different countries. Another issue with KPIs became apparent when exploring the application of KPIs within NEXES. Can we use one set of KPIs for all emergency services? This makes sense: by defining one (albeit) large set of KPIs, it becomes possible to assess the impact of the new Internet-enabled technologies, compare the new performances of emergency services and ultimately give compliments to emergency services that implement these new technologies.

However, soon it became apparent that the ‘one size fits all’ approach was not deemed useful at all by emergency services. As a result, the “one-size-fits-all” approach to KPIs was abandoned. A creative solution is needed, one that is fair, respectful and motivating.

The standard structure of one KPI is to measure a single effect accompanied by a certain target value, e.g. “arrival time of first responders at incident location should be less than 10 minutes”, or “average call waiting time should be less than 10 seconds”. The situation unfolds that traditional KPIs are not easily applied to multiple, heterogeneous organisations. Emergency services are organised differently and each has their own plan for implementation of NEXES solutions. So, how can a KPI be defined more flexibly so that it can accommodate the differences between emergency services?

NEXES solution lies in decoupling effects from KPIs, by describing a common denominator in the KPI which is to be specified in detail in the effect measurements by the single, different parties. A flexible KPI is in practice based on multiple effect measurements by the partaking organisations. And the relevant effect measurements can be tailored to their specific situation.

Within the NEXES Action, we used this idea to define flexible NEXES KPIs. First, a set of 11 KPIs is defined and grouped into categories ranging from Emergency Calls to the Coordination of Response Activities and Customer Satisfaction, where each KPI is mandatory. That means each emergency service must have a score for all KPIs. A KPI is not anymore defined as a single effect, rather each flexible KPI is dependent on multiple effect measurements, currently 58. The score of a KPI can be calculated by aggregating the scores for the relevant and tailored effect measurements.

The approach chosen by NEXES is thus to guarantee the autonomy of emergency services. Here, ‘autonomy’ refers to the ability that an emergency service governs itself, makes its own plans, and decides when and how to adopt specific NEXES solutions. Any approach that limits the autonomy of emergency services is destined to fail.
The autonomy of emergency services is guaranteed in two ways. First, by requiring any emergency service to select those effect measurements that are relevant to their situation. Second, by allowing an emergency service to tailor the relevant effect measurements to their specific situation. Furthermore, NEXES requires that all of these decisions and adaptations are made visible. Having these aspects visible makes it possible to compare emergency services’ adoption of NEXES solutions: the original reason for introducing KPIs.

Within NEXES, an effect measurement has a structured description. An emergency service is invited to ‘fill in the blanks’, thereby tailoring the effect measurement to their situation. For example, consider effect measurement #31, which measures Incident Location Accuracy. This effect measurement measures the discrepancy between the incident location determined by the emergency services’ operators versus the actual location of the incident. It is expected that using GPS-based location information from callers, the incident location becomes known more accurately than using the cellular base-stations triangulation algorithms, especially in more rural areas.

Furthermore, emergency services can specify how important certain effect measurements are when calculating their KPIs’ scores. The mechanism provided for this, is that the score of an effect measurement gets an associated importance before aggregation into the KPI score.

For each emergency service, we cannot ‘equalise’ the measurement methods: that would bring us back onto the “one size fits all” path. The measurement methods of the emergency services will be assumed to be different, although these may be similar. Based on the resulting measured values, the evaluation by emergency services is tailored to their own needs. This tailoring is what we can re-use: we can use the tailoring of any one emergency services and apply it to itself (e.g. after a change) or another emergency service (e.g., out of curiosity). So we can compare the KPIs’ scores from the perspective of any emergency service by applying tailoring to many emergency services including itself after a change. It also becomes possible to re-compute the scores for all emergency services, given new insights in e.g. the relative importance of specific measurement methods, or using similar reference values, or any other reason.

The flexibility of the KPIs and effect measurements has shown to be beneficial (1) to guarantee autonomy of emergency services in the application and tailoring of the KPIs, (2) to enable structured comparison of results of the before and after situation of an emergency service, and (3) to foster exploration and understanding of the NEXES impact by changing and re-computing scoring formula and scores of KPIs for (all) emergency services.
NEXES’s challenge was to find an alternative approach to replace the ‘one size fits all’ KPI-approach. NEXES adhered to the ‘design for change’ adage and created the principle of flexible KPIs. This flexibility is achieved through the tailoring of relevant effect measurements and scoring importance.

Is the current set of NEXES effect measurements, 58 in total, finished? Not likely. The NEXES Action is set to conduct a number of exercises in which these KPIs will be applied to emergency services in Slovenia, Romania and Turkey that implement some or all of the NEXES Internet-enabled technologies. These exercises will also yield insight in the usefulness and practical applicability of the current KPIs and effect measurements. It may even lead to changes to the KPIs: that is a welcome effect of accumulating progressive insight. The net result will be an improved set of flexible KPIs that is useable by emergency services throughout Europe and beyond.

Moving forward, NEXES will gain experience with applying flexible KPIs and explore the opportunities to improve the current NEXES KPIs and effect measurements to the benefit of next generation emergency services across the world.

NEXES Attains Its Second Milestone !

The NEXES Testing Regime and Validation Framework Sets New Advances Towards Next Generation Emergency Services

The NEXES Consortium announces the attainment of the Second Milestone in their research and innovation effort: the establishment of the NEXES Testing Regime and Validation Framework brings NEXES a step closer to making next generation emergency services a reality.

At the Milestone 2 Meeting held in May 20th 2016, the NEXES partners unanimously agreed that the present status of NEXES warrants the reach of Milestone 2, as the Consortium defines the first iteration of the NEXES open Testing Regime and Validation Framework, considering the end-to-end communication between citizens and emergency services. The NEXES Testing Regime and Validation Framework aims to empower the assessment of the value and benefits of next generation emergency services as they embrace total conversation, enhanced location and improved interoperability capabilities. The NEXES partners also acknowledged that this successful evaluation of the NEXES effort has been possible because of the work performed during the last year, creating the user requirements for the next generation emergency services, defining the pan-European standard for emergency Apps and drafting the initial NEXES Exercise plan.

“The NEXES Consortium is proud of its leadership role in the development of next generation emergency services. NEXES, a reference implementation of next generation emergency services, is valuable for first responders, emergency services and citizens,” said Marco Manso, Director of Innovation of RINICOM and NEXES Coordinator. “Our advanced Testing Regime and Validation Framework is a key component of NEXES to transform emergency services. We are gathering feedback and contributions from our end-users as we develop our solution for next generation emergency services and we truly believe that NEXES will help citizens to reach emergency services in case of need and assist the performance of emergency services.”

milestone2_b milestone2_a

 

International Experts Assemble at the NEXES Workshop on Accessibility

Vivid conversation on next generation emergency services in a room full of silence

The NEXES Workshop on Accessibility held on May 19th addressed the key questions and challenges associated with the design and development of accessibility features for next generation emergency services.

20160519_160540

Organised by the European Union of the Deaf and OMNITOR, the Workshop allowed international experts from across Europe and Turkey to understand the NEXES vision for truly universal, democratic and inclusive emergency services. The experts reviewed specific case studies where accessibility to the deaf and hard of hearing communities has been adopted and discovered the many benefits of the CAT112, the emergency App from the Catalonia Emergency Service that renders these services accessible for citizens with speech impairment and hearing disabilities by using multi-channel communication.

20160519_153933 20160519_152959

Drawing on these cases, experts shared their own personal experiences witnessing the attacks in Brussels and London and earthquakes in Japan and focused on the issues of communication barriers and lack of information. They combined their expert knowledge to engage in a fictional Emergency Exercise that encouraged the vivid discussions to explore and find solutions to issues currently challenging accessibility in emergency services in Europe and beyond.

IMG_20160520_125431

The NEXES Workshop on Accessibility is part of the overall dissemination and engagement process towards the adoption of next generation emergency services that make accessibility a top priority. Work and results from the meeting will feed into the NEXES RIA development process and future implementation of the NEXES System and Apps.

NEXES_Omnitor_EUD

NEXES Presented to the European Deaf Community

Dovenschap Interested in NEXES Proposal to Create Inclusive Emergency Services

6a311467acf1cfabd1677f6e8ef967b1_f2132

The NEXES Consortium was formally invited by the European Union of the Deaf (EUD) and the Dutch Association of the Deaf, Dovenschap – doof, slechthorend, doofblind & gebarentaal, to present the Action’s innovations and vision to create inclusive emergency services at the Associations’ Seminar 2016, at the Europa Hotel Scheveningen, in The Hague, on May 20th.

17851e4afda8c12c27c55d1f592d124b_f2113

The EUD/Dovenschap Seminar 2016 dedicated a full day to the discussion of European Deaf Innovations: Best Practises and NEXES starred as the top European research and innovation action addressing the needs of the deaf, hard of hearing and hearing impaired communities in the framework of next generation emergency services. Led by Rinicom, the NEXES Action has the European Union of the Deaf as a main partner and the NEXES Accessibility Advisor, responsible to issue recommendations on the accessibility features of NEXES and to establish fruitful synergies with the deaf, hard of hearing and hearing impaired communities across Europe.

e9e180837339a31e910e55a30f3b8be7_f2133

The Action’s achievements were briefly presented to an audience of about one hundred delegates from across Europe, with a special highlight to the incorporation of the NEXES’s innovations in total conversation, improved user interface and automated messages and alerts into a new emergency App for citizens that provides a direct link to the right emergency service and brings citizens experiencing physical disability and special needs to the frontline of the dialogue for a universal, democratic and inclusive emergency service in Europe.

The intervention at the EUD/Dovenschap Seminar 2016 provided the NEXES Consortium with a formidable opportunity to interact with an experienced audience, specifically envisaged in NEXES’s targeted dissemination effort, which aims to create awareness and engage relevant stakeholders to the development and adoption of NEXES’s innovations across Europe.

2258d977004ba7dcebcaa0eaf48c1917_f2114

NEXES Exercise on Accessibility

On May 19th 2016, during the NEXES Workshop on Accessibility, the NEXES RIA began its Campaign of Demonstrations, hosting its first Exercise, dedicated to the Accessibility in emergency services in general and the Accessibility features in NEXES, in particular.

The presence of a large audience of members of the deaf and hard-of-hearing community presented an excellent opportunity to engage directly with members of the deaf community and to secure valuable feedback on their experience dealing with emergency services and on the enhancements considered by NEXES.

Organised by EUD, the exercise took the form of a ‘walkthrough’ of an emergency response scenario involving a terrorist attack on a mass transit system (NEXES Reference Scenario 3), delivered in sign-language by representatives from EUD and OMN, with designated intervals for facilitated group discussion. The method used for the table top exercise presented no logistical constraints to the scale of the scenario. Being large-scale and complex, this scenario also gave the opportunity to explore the widest possible range of NEXES functionality and end-user requirements. The following images helped to set the ambiance for the exercise.

The NEXES Exercise on Accessibility counted with 38 participants from across Europe, each presenting deafness or hard-of-hearing impairment. The statistics on the participants show:

Discussion topics focused therefore on currently available communications options and any associated impediments to communication, as well as the proposed alternatives considered by NEXES, and their advantages and/or limitations. Topics were not overly proscriptive however, and free and creative discussion was encouraged, and enthusiastically engaged in. The exercise scenarios were written from a variety of perspectives (e.g. victim, bystander, concerned citizen), which provided a fruitful means of encouraging participants to think about the topics in the context of the broadest possible range of situations.

Organised by EUD, the exercise took the form of a ‘walkthrough’ of an emergency response scenario involving a terrorist attack on a mass transit system (NEXES Reference Scenario 3), delivered in sign-language by representatives from EUD and OMN, with designated intervals for facilitated group discussion. The method used for the table top exercise presented no logistical constraints to the scale of the scenario. Being large-scale and complex, this scenario also gave the opportunity to explore the widest possible range of NEXES functionality and end-user requirements. The following images helped to set the ambiance for the exercise.

Participants’ feedback to the NEXES Exercise on Accessibility, and in consideration of the proposed NEXES capabilities, brought about considerations that may not be readily apparent to non-deaf individuals. For example, the fact that deaf people rely on visual communication, regardless of whether they have sign language as their first language or whether they primarily lip-read. The feedback on ensuring a highly visual intuitive user interface to the NEXES App is also relevant to communicate with tourists and early migrants who cannot speak the native language. It also highlighted that accessibility is not simply a matter of developing solutions targeted at specific user groups, but rather needs the promotion of universal accessibility by employing technologies and processes which facilitate access for the greatest range of users.

Participants also emphasised the need for the NEXES App to provide accurate information sent by emergency services, the ability to verify the information’s provenance and its update capability, as well as made relevant suggestions for simple NEXES App functions, such as the utility of a ‘one touch’ system, accessible when a device’s screen is locked, the value of tutorials on emergency preparedness and first aid, the relevance of clear mapping highlighting areas of danger/safety and the opportunity to include a signal/beacon to alert the deaf of the emergency services’ presence. In addition, concerning the deaf community, participants reinforced that the challenges of communicating with emergency services persist after initial contact, throughout subsequent interactions with First Responders, since it is obvious the lack of training among emergency services personnel on how to communicate with the deaf population.

All the findings and conclusions retrieved from the NEXES exercise participants’ feedback are an integrated part of the overall NEXES user requirements and are being duly considered in the development of the NEXES App for citizens.

NEXES at the 2nd Pan-Hellenic Congress of Emergency Pre-hospital Care

NEXES Novelties for Emergency Management Systems

NEXES was presented by the Ambulance and Emergency Physicians Association (AAHD) at 1-76caf93dc3the 2nd Pan-Hellenic Congress on Emergency Prehospital Care on “Emergency Pre-Hospital Care: Present & Future”, which took place in Thessaloniki from May 11th to 15th 2016 at the Porto Palace Hotel. The Congress was organised by the Hellenic Society of Emergency Pre-hospital Care and the Cyprus Resuscitation Council and gathered more than 500 participants, discussing the whole emergency care and emergency medicine family. During the Congress, it was also possible to attend to the Hellenic-Cyprus Resuscitation Symposium that received the participation of the foremost members of the European Resuscitation Council (ERC) to discuss the modern developments in the field of Cardiopulmonary Resuscitation, including the New Guidelines published in 2015.

Before an audience of first responders, volunteers, members of security forces, paramedics and specialised clinical practitioners, AAHD made a presentation titled “Emergency Management Systems of the Future” and announced NEXES’s proposed innovations to the design and implementation of next generation emergency services when addressing trauma in prehospital environments and confronting the challenges of mass casualty incidents.

NEXES’s participation at the 2nd Pan-Hellenic Congress on Emergency Prehospital Care is part of the motivated dissemination and communication effort of the NEXES RIA, with the purpose of building awareness on NEXES and promoting its contribution to the long-term strategy to reform and reconstruct Emergency Prehospital Care and Emergency Medicine, before a public that is renowned for contributing to the evolution in the world of emergency, through their research, daily clinical work and ethos.

zeynep

NEXES Hosts a Workshop Dedicated to Accessibility

In Debate A Fully Universal, Democratic and Inclusive Emergency Service

The NEXES Consortium will be hosting the NEXES Workshop on Accessibility on May 19th 2016 at the Bilderberg Europa Hotel Scheveningen, in Scheveningen, The Netherlands.

This Workshop signals the NEXES Consortium’s effort to create a reference implementation of the next generation emergency services, upholding accessibility principles and following a truly universal, democratic and inclusive approach.

The NEXES Vision on Accessibility in emergency services and the sharing of experiences and best practices are therefore the proposition we bring forth to you during this Workshop, aiming to establish a true dialogue forum, where your presence is invaluable.

For the Workshop’s agenda, please see it here.

If you wish to join us at the NEXES Workshop on Accessibility, please email us at coordinator@nexes.eu.

nexes_poster