zOS TWS performance problems - Nordic TWS conference

Nordic TWS conference Copenhagen 2011TWS/z performance problems2 Issues12/05/2011 Torkild Busk and Steen Kasbæk © 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Issues for an installation that run 80.000 jobs daily Poor performance in dialog TWS submit jobs very slowly Jobs remain in status R for several minutes Even ”dummy” operations in status R didn’t start TWS controller task was not busy for long time2 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Solution If large dataset in EQQJBLIB, JCL library must be in LLA Event filtering must be implemented Replan MUST be considered Reduce number of jobs with status A,R and * Avoid upload of data in AD/CP/OI when batch peakload3 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions orImprovements– Large JCL libraries withmore than 50.000members MUST bestored in LLA with”freeze”. What could be changed– Define a member including name oflibraries that should be freezed in LLAjust before night batch and a memberto remove the libraries from LLA in themorning.– When freezed, JCL ”can not bechanged”– Define two jobs4 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvementsMember CSVLLATLLIBRARIES(PRODC.OPCA.JOBJCL, PRODC.OPCP.N.JOBJCL)FREEZE(PRODC.OPCA.JOBJCL, PRODC.OPCP.N.JOBJCL)Member CSVLLATRREMOVE(PRODC.OPCA.JOBJCL, PRODC.OPCP.N.JOBJCL)JCL//SDSF EXEC PGM=SDSF//ISFOUT DD SYSOUT=*//ISFIN DD */RO *ALL,F LLA,UPDATE=TR/RO *ALL,F LLA,UPDATE=TL5 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Event filtering wasimplemented. SMF / Jes exitsupdated, so event is writtenonly if trackername is firstcharacters in programmername of the job. What could be changed– Submit exit EQQUX001readASCBJBNS field– ’XOPC-+’ should be firstcharacters in programmername of the JCL.– Exit substitude XOPC-+ withtracker that should track thejob.– The exit can be used bydifferent controllers in thesame SYSPLEX environment6 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– REPLAN is executed at leastonce a day. What could be changed– REPLAN remove all completedoperations.– Reduce any search, update orlist if running e.g. PIFprograms– List in option 5.2 or 5.3 ”will befaster”7 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Virtual WS implemented What could be changed– Virtual WS can improve yourperformance.– Be sure that Dormancy valueis correct.8 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements9 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Reduced the number ofoperations in status A, R and * What could be changed– Reduced the number ofoperations in status A, R and *– Create an application for eachhour to release time dependentjobs– Jobs with submit=no– Special resources not available– ”AUTO” work stations G/A10 12/05/2011© 2011 IBM Corporation



TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– ETT table reviewed What could be changed– Avoid entries with * or %14 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Increase the region size in thecontroller task. What could be changed– Increased region size to 0M15 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions or Improvements– ParametersWhat could be changed– Parameters• CPDTLIM(100)• CPBPLIM(40)While the CONTROLLER is active, the CURRENTPLAN is LSR buffered for performance. AT ALL TIMES,100% of the INDEX CLUSTER and 50% of the DATACLUSTER of the active CURRENT PLAN dataset is INMEMORYFor best performance, settingCPDTLIM(100), CPBPLIM(40)is recommend, and adjusting theCONTROLLER REGIONIf necessary to allow 100% buffering of theCP. (Many high volume users setREGION=0). See PMR 50205,010,678• QUELENLEN(20)• STATIM(60)16 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvementsEQQE000I TOTAL NUMBER OF EVENTS PROCESSED BY THE EVENT MANAGER TAEQQE000I NUMBER OF EVENTS SINCE THE PREVIOUS MESSAGE IS: EQQE000I EVENT MANAGER QUEUE LENGTH STATISTICS FOLLOW: EQQE000I TOTAL Q1 Q2 Q5 Q10 Q20 Q50 Q100 EQQE000I 13229 12456 220 244 144 89 53 20 EQQE006I EVENT MANAGER EVENT TYPE STATISTICS FOLLOW: EQQE006I TYPE NTOT NNEW TTOT TNEW TAVG NAVG EQQE007I ALL 391272 19913 315.4 15.8 0.00 0.00 EQQE007I 1 55608 2813 23.0 1.1 0.00 0.00 EQQE007I 2 46125 2563 17.8 0.9 0.00 0.00 EQQE007I 3S 5727 620 2.1 0.1 0.00 0.00 EQQE007I 3J 46099 2538 13.7 0.6 0.00 0.00 EQQE007I 3P 55673 2813 124.0 9.3 0.00 0.00 EQQE007I 4 0 0 0.0 0.0 0.00 0.00 EQQE007I 5 55258 2622 0.0 0.0 0.00 0.00 EQQE007I USER 3101 147 4.8 0.2 0.00 0.00 EQQE007I CATM 240 10 9.4 0.3 0.03 0.03 EQQE007I OTHR 116724 5739 53.2 2.9 0.00 0.00 EQQE007I FT 3222 0 65.9 0.0 0.02 0.00 EQQE007I ZCEN 864 0 1.1 0.0 0.00 0.0017 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Truobleshooting and suggestions/improvement18 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 201119 12/05/2011© 2011 IBM Corporation



TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– JCL debug stats What could be changed– F xxxx,JCLDBG=ON– F xxxx,JCLDBG=OFFWhere xxxx is the CONTROLLERstarted task / subsystem nameFurther informations can be found at:https://www-304.ibm.com/support/docview.wss?uid=swg2125280622 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Be aware of extend daily planwhen you have a high amountof TWS jobs running What could be changed– Rescheduled extend ofcurrent plan, if high numberof TWS controlled jobs isexecuting combined withhigh load at the SYSPLEX.– EQQW016E Anunrecoverable I/O erroroccurred23 12/05/2011© 2011 IBM Corporation

TWS Nordic conference Copenhagen 2011Troubleshooting and suggestions/improvements Suggestions/Improvements– Be aware of job that interactwith the TWS controller What could be changed– Rescheduled batch jobs to nonepeak load periode• Unload from AD / CP• Load of OI in CP• Delete of ”old” JCL in JSPGM=JSDEL• Insert of predecessors• Joblog retrieval jobs24 12/05/2011© 2011 IBM Corporation

zOS TWS performance problems - Nordic TWS conference

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?