Operational Efficiency Workshop Evaluation Report - Emergency ...

Emergency Department:Operational Efficiency WorkshopEvaluation ReportIntroductionOn the 29th March 2012, the NSW Ministry of Health and the Emergency Care Institute NSW (ECI) held aworkshop for ED clinicians and managers on Emergency Department (ED) Operational Efficiency. There were148 people registered for the event and 131 attended from a range of disciplines and facilities.Design and PurposeThe workshop was designed to look at how to improve the efficiency of the ED, particularly with the upcomingchanges with the introduction of the National Access Emergency Target (NEAT) from 1 January 2012 andintroduction of Activity Based Funding (ABF) in NSW from 1 July 2012.There was a particular focus on providing further information and generating discussion on:• Operational flow and the efficient ED• Communication efficiency• Collaboration and culture• Using objective measurement to drive process changeAll participants were provided with an evaluation form and 99 were returned, a response rate of 76%. Theinformation below summarizes the feedback from these respondents.FeedbackRespondent’s organization and positionFeedback was received from respondents in all Local Health Districts (LHDs) excluding Far West LHD and StVincent’s Health Network. The bulk of respondents providing comments associated themselves with their LHD.Those that stated their hospital, were from a range of Level 2,3,4,5 and 6 facilities.Number of resondents1614121086420LHD of respondentsThe role delineation level of thosestating their facilityn=35n=2n=1n=6n=16n=1223456LHDn=18Othern= 90n= 901

The majority of those providing feedback were in the role of either NUM, CNC or NM (n=48) with 12 EDDirectors providing feedback.Current positionNumber of respondents35302520151050n= 92The workshopNone of the respondents rated the workshop as either Fair or Poor with all rating it positively. 79% rated it aseither Excellent or Very Good. The overwhelming majority of respondents felt that the workshop was pacedappropriately (92%).Workshop ratingn=18Very GoodPace of workshop n=1n= 6Appropriaten=26n=42ExcellentGoodn= 83Too slowToo fastn=86n= 90Over 88% felt the workshop met their expectations, none disagreed that it was well planned and 90% believedthat it gave them new tools to implement in the ED. One person felt that it did not provide them with tools totake back and implement in their ED and 3 did not believe that it met their expectations.ACI/D12/28332

The workshop/day overall:100%90%80%70%60%50%40%30%20%10%0%met my expectations was well planned and organised provided me with new information /tools to try and implement in myEDStrongly agree Agree Unsure Disagree No answern= 99The venue itself was rated positively with no one viewing it as poor although there were concerns raised aboutthe location, tables feeling somewhat cramped and the temperature of the room being cold. The catering wasrated highly with 85% of respondents believing it to be very good or excellent. There was no qualitativefeedback on this. This was also true of the presenters with over 80% rating them as very good or excellent.The audiovisuals received some negative ratings with 5 respondents viewing them as fair or poor. The onlyqualitative comments received in relation to this were regarding the difficulty in seeing the whiteboard duringthe afternoon interactive workshop on measurement and change.100%80%The supporting aspects of theworkshop wereWould you like a follow up eventorganised?n=160%40%20%0%n=14YesUnsureNon=72Excellent Very Good Good Fair Poor No answern= 99n= 8773% of respondents would like to see a follow up event organized in the future. Only 1 person did not wishanother event to take place. Those unsure gave feedback including that the workshop should focus more onwhole of hospital, not just ED as well as being more focused on the Australian healthcare system as opposedto looking at the US.ACI/D12/28333

The sessions and content of the dayRespondents were asked to rate each of the sessions that took place. The most popular sessions, based onthe amount of excellent or very good ratings, were:• Communication and Efficiency• Operational Flow• Collaboration and CultureThe Activity based Funding (ABF) and NEAT sessions were rated less highly although they were still regardedas excellent or very good by over 50% of the respondents.The session was:100%90%80%70%60%50%40%30%20%10%0%NEAT ABF Operational Flow CommunicationEfficiencyCollaboration &CultureMeasurements &ChangeExcellent Very Good Good Fair Poor No answern= 99The majority of respondents felt that all relevant issues were discussed although it should be noted that therewere areas that respondents felt could be expanded, particularly in relation to making the day more Australianfocused, there being too much information for one day and people wanting different levels of information.Do you feel all relevant issues were discussed?Yes74%Maybe7%No19%• We need more of these workshops to address combined ED issues which will reduceED stress, staff satisfaction, staff retention and therefore patient satisfaction• A well focused day• Appropriate for time allocated• Quite well addressed• Whole system approach has to be taken, not just the ED• View that the discussions were not relevant to Australian healthcare system as focusedon US with little understanding of our system.• More info on what's worked well - give examples• Overall emphasis on healthcare as a service industry, as US based. Need measurableoutcomes that justify this approach i.e. improved health of target population• Too big a topic for one day• More NEAT• Too much talk about scribes.• Not all information was new information• The day should have been more dedicated to practical issues e.g. registrar review andACI/D12/28334

specialist input• Too much emphasis on Dr Dan Smith• Lack of answers about ABFWhat respondents felt was of most and least benefit from the workshop?In line with the ratings that respondents gave to the sessions they found that the Operational Flow andCommunication and Efficiency sessions were of most benefit. Surprisingly, the collaboration and cultureworkshop was not mentioned to the same extent despite being rated highly.Number of respondetns302520151050Which part of the workshop did you benefit most from?* On occasion, respondents stated more than one answern= 106*The sessions the respondents got least benefit from were in line with the ratings they provided for eachsession. NEAT and ABF were mentioned the most in terms of not being of benefit.Number of respondents181614121086420Which part of the workshop did you benefit least from?* On occasion, respondents stated more than one answern= 63*ACI/D12/28335

The individual comments relating to the aspects of the workshop that the respondents found most beneficialare included in Appendix I. These include the following generic aspects:• Practical ways of supporting EDs• Relevant to EDs• Achievable ideas without significant resource• Building on existing knowledge• AIDET was particularly beneficial to respondentsThe detailed concerns that respondents had are also highlighted in Appendix I and include:• Nothing new bring presented that hadn’t been heard before• Not relevant to Australian healthcare• Not enough information or answers providedOther commentsRespondents were asked to provide other comments which have been categorised.Do you have any other comments about any aspect of the workshop?Number of respondents9876543210* On occasion, respondents stated more than one answern= 44*The full comments received from the respondents are provided in Appendix II.ConclusionOverall all respondents rated the workshop and sessions very positively with those attending finding it met theirexpectations, was well organized and provided tools for them to take back to their ED. The pace wasoverwhelmingly felt to be appropriate and the ability to be able to network with colleagues was commented onthroughout.Respondents valued the practical workshops and tools that they could take back to their workplace whereasthe more generic theory based sessions were viewed as less useful. This should be considered in futureevents where there might be more use of communicating key information in other formats e.g. newsletters, prereading etc. therefore leaving the day focused on more interactive, practical sessions.ACI/D12/28336

The difficulties in tailoring a workshop to different knowledge bases was apparent in many of the commentswith some respondents feeling that they had already been involved in previous discussions around similartopics whereas for others it was very welcomed.The venue was regarded well although there were concerns raised about the location due to the difficulties inreaching Parramatta, especially from regional attendees. The fact that so many attendees were from theSydney or surrounding LHDs would perhaps reinforce this.The speakers were well regarded with Dr Dan Smith, who led multiple sessions, receiving numerous positivecomments relating to his engagement and enthusiasm.A further workshop would be welcomed with the opportunity for more people to attend. In organizing this, theissues that have been raised including location, applicability to Australian healthcare, level of information beinggiven and practical tips for improvements should be taken into consideration. Consideration should also begiven to opening this up to other disciplines and involving the wider health system, given the concerns raisedby attendees that many of the issues are achievable only with the whole of hospital/health system approach.ACI/D12/28337

Appendix I – Which part of the workshop did respondents benefit most and least fromOperationalFlowCommunicationEfficiencyOtherAll of Dr DanSmith'spresentationsActivity BasedFundingMeasurements& ChangeACI/D12/2833Which part of the workshop did you benefit fromMostLeast• Allowed an opportunity to identify areas of • Already doing it so no new informationimprovement that would provide most impact.• New, useful and relevant tools to identify areasof improvement• Practical ways to improve efficiency in the ED• Implementing AIDET• Practical models of care• Ideas without always needing significantresource• Achievable and transferrable ideas• Ideas that will really help us• Added depth to existing knowledge• AIDET tool discussion and explanation• Easiest to adopt and implement• Easy to adopt• Generation of great ideas to developcommunication and team work in the workenvironment.• Achievable strategies• Opportunity to discuss with ED team sittingalongside away from the coalface• Idea of results pending area and freeing up EDbeds• Increased motivation to keep on trying toimplement changes• Useful strategies for getting team membersconvinced and motivated• Benefits easily seen• Different slant on things already known• Enthusiastic, dynamic, passionate, wellarticulated presenter• Exposure to new ideas and approaches• Practical ideas that can be implemented withoutmuch trouble.• Greater understanding• Questions were clarified around how thisimpacted EDs• This is a very current topic• This will help me respond to staff about ABF• Nothing new, heard it all before• Need new strategies - everyone is used tothese• I have heard AIDET tool previously• Need to fix the back end first.• All this has been previously done –nothing new• Discussion was about things we alreadyknow• USA a different system to Australia• Too much focus on scribes• Not relevant to me• Left many questions unanswered• Content rhetorical with no cleartimeframes• Have heard this before• Concern being implemented withoutclinician involvement• Gave practical ideas • Hard to see the board• Hard to generalize as only onedepartment was being discussed• LHD is required to buy in• There was not enough representationfrom my unit to make discussionworthwhileNetworking • Able to networkNEAT• Applicable subject and to the workplace• Learnt information relating to this• Informative, inspiring talk• Already informed on NEAT• Too positive about NEAT• Have heard this all before8

Collaboration &CultureAllpresentationswere useful• Would have liked to hear about resourcesavailable• No new information provided• Not all facts could be presented due totime restraints• Not pertinent to today's otherpresentations• No comments • Already done• Old news - unsure of purpose of thesetalks• All components provided valuable informationand self reflection on overall practice• All different and relevant to hear new strategiesand stresses on ED• Fact it related well to me• Relevant to ED practice• All presentations were beneficial• I felt all the topics had some relevanceACI/D12/28339

Appendix II – Other comments received from respondentsThemeToo US focusedOther commentsWhole of system - notjust EDSpeakersLocationVenueNetworkingPositive thanksMake information fromthe day availableFuture workshopsACI/D12/2833Comments• The USE and AU systems are different.• There needs to be more collaboration and networking from within NSW and Australia• Good tools for change• ED staff need to identify their part in many of the problems faced and discussed today: notblaming funding or patients.• Clinicians need more support• Too much talk about scribes. Very good. Will attend more• It will be great to see the progression and innovation of individual EDs• After lunch didn't work for me. Couldn't see board - trying to do a small group activity as abig group just lost my interest.• Need to implement pay for performance model similar to US for medical practitioners, withscorecards for Quality, complaints etc. If wish to implement their strategies, as per previousstrategies implemented half heartedly, they don't work and breed discontent with theclinicians hence further resistance to change.• Given the people (RN's, MO's) present there could have been other useful information.• Could have spent more time on hospital wide strategy to achieve NEAT 4 hours e.g. howto engage inpatient team• I think the message needs to come across that NEAT requires a whole of hospitalapproach to be successful and I don't think this came across. I know it’s an ED focusedworkshop but it’s an important point to be aware of.• Dr Dan Smith great presenter• I thought Dr Dan was great and the group was a little defensive and precious at first but theday was well worthwhile.• Dr Dan's presentations were excellent and give the ability to look at system improvements.• Well thought out and excellent speakers• Given the people (RN's, MO's) present there could have been other useful information.• Presenter is an excellent captivating speaker.• Dan Smith is an excellent speaker• Dr Dan Smith provided overall motivation for change and self reflection in my workplacepractices.• Ensure funding to future events as no travel or accommodation to be approved for our LHD• Difficult location• Have the workshop in a more rural location• Tables cramped in room, room very cold in the morning. People standing around atmorning tea so no room to move.• Good spot and great parking• Good networking.• Good networking• Very enjoyable and great networking and catching up with old friends and colleagues• Well organised. Well done team• Thank you• Excellent in general thanks• Very constructive• Thank you very much• Useful workshop, thank you.• Thank you• Well done! Lots of food for thought.• Could we please put the talks on ECI homepage for access• PDFs of Struder stuff• Presentations put on website• Pity the opportunity was limited to c.75 places. I was aware of many who missed out onthe opportunity• To ensure that as an executive sponsor I can ensure other key members of staff can10

attend. Collaboration and culture needs to be mandatory for all senior medical doctors inALL specialties.• Another workshop about MOC• Follow up with ABF answers when become clear• Please keep these sessions going. Much benefit in exposure by all levels of staff - wouldlead to increased staff engagement.• More workshops about practical tools for EDs which hear others say as well.ACI/D12/283311