OMVS latch deadlock –

Unix System Services 

OMVS latch deadlock – 

detected on the second view 

GSE z/OS Guide Lahnstein, 03/14/2012 

Matthias Korn 

z/OS Virtual Front End 

IBM Deutschland GmbH 

matthias.korn@de.ibm.com 

© 2012 IBM Corporation

IBM Global Competency Center 

The Dining Philosophers Problem 

Quelle: Wikipedia 

2 



What is a Deadlock 

in use by 

Process / Task A 

requires 

Resource A 

Resource B 

requires 

Process / Task B 

in use by 

3 



Details from the SADUMP of MVSS: 

IP ANALYZE RESOURCE 

RESOURCE #0133: 

NAME=SYS.BPX.A000.FSLIT.FILESYS.LSN ASID=0010 Latch#=2 

RESOURCE #0133 IS HELD BY: 

JOBNAME=OMVS ASID=0010 TCB=009A1050 

DATA=EXCLUSIVE RETADDR=A347EFD2 

REQUEST = 11/15/2011 17:36:15.057144 

GRANT = 11/15/2011 17:36:15.142437 

4 



Details from the SADUMP of MVSS (cont.): 

RESOURCE #0133 IS REQUIRED BY: 

JOBNAME=OMVS ASID=0010 TCB=009DA188 

DATA=EXCLUSIVE RETADDR=A3796972 

REQUEST = 11/15/2011 17:36:15.144127 


DATA=EXCLUSIVE RETADDR=A36A6452 

REQUEST = 11/15/2011 17:46:00.160780 



REQUEST = 11/15/2011 19:44:17.379913 

5 




IP OMVSDATA DETAIL ASID(16) 

Thread Data: Active Threads 

Thread ID: 23A2E000 00000000 

Tcb Address: 009A1050 

Space switch stack: 

IP CBF 7A3F8000. ASID(X'0010') STR(BPXSTACK) 

WorkerThread Data: 

Requesting ASID: 0100 

Requesting System: 5



IP CBF 7A3F8000. ASID(X'0010') STR(BPXSTACK) 

BPXNXWRK -> BPXNXWRK(BPXNXWK1) -> BPXTXRXS -> 

BPXFVNL -> BPXTAVNO -> BPXFSMNT -> BPXLKKW 

7 




IP WJSIP -> FILESYS 

>>> latch 2 held Exclusive has waiters 

ASID=0010 TCB=009A1050 Caller=2347EFD2 LQE=7E67D4F0 EP=BPXNXWRK 

>Waiters: 

Stok=00000040 TCB=009DA188 Caller=23796972 LQE=7E67D5C0 EP=BPXFSLIT 

Stok=00000040 TCB=009D4200 Caller=234FB25E LQE=7F4B01B0 EP=BPXNXWRK 

Stok=00000040 TCB=009D3528 Caller=2344F780 LQE=7E2956F8 EP=BPXNXWRK 

Stok=00000040 TCB=009CF0A8 Caller=236E6096 LQE=7F4B06F8 EP=IGWLQDTT 

Stok=00000040 TCB=009D43B0 Caller=23459608 LQE=7F4B09D0 EP=BPXNXWRK 

Stok=00000040 TCB=009D0D78 Caller=236A6452 LQE=7E2FA148 EP=BPXNXWRK 

Stok=00000040 TCB=009A1260 Caller=236A6452 LQE=7E67D148 EP=BPXNXWRK 

Stok=00000040 TCB=009CFCF0 Caller=2344F780 LQE=7E2954F0 EP=BPXNXWRK 

>>> latch 442 held Exclusive 

ASID=0010 TCB=009A1050 Caller=23480E62 LQE=7E2FA078 EP=BPXNXWRK 

8 




IP BPXWFLCB 442 

***************************************************** 

** ** 

** BPXZFLCB INDEX LOOK-UP ** 

** ** 

** ALL VALUES IN HEX UNLESS OTHERWISE STATED ** 

** (DEC SPECIFIES A DECIMAL VALUE) ** 

***************************************************** 

RELEASE IS: HBB7770 

LATCH NUMBER (DEC): 442 

LATCH NUMBER (HEX): 1BA 

THIS LATCH NUMBER REFERS TO EYE CATCHER: AVFS 

FILE SYSTEM NAME: *AMD/u$/tu 

EXECUTION FINISHED 

9 



Conclusions / Questions at this point 

• Processing got stuck during (auto-)MOUNT 

• No deadlock could be identified so far 

• WAIT was entered in BPXFSMNT – main mount entry point 

• What delayed the (auto-)MOUNT 

• What is BPXFSMNT waiting for 

• Is it waiting for any resource 

• Is it waiting for any subtask to respond 

10 



BPXFSMNT 

• Entered the WAIT in a routine called GetSecLabel 

• SECLABEL is a RACF class, that allows to set security labels on z/OS UNIX resources 

• Creates and queues a work element and posts BPXFSLIT 

to obtain the security label for the file system to be mounted 

• waits for BPXFSLIT to return the security label 

What is BPXFSLIT doing 

11 



How to find the BPXFSLIT task 

IP SUMMARY REGISTER ASID(16) 

F BPXFSLIT 

TCB: 009DA188 

CMP...... 00000000 PKF...... 00 LMP...... FF 

General purpose register values 

0-3 80000314 7FFBB028 000A5A7D 08069288 

4-7 00000000 00000000 00000000 00004200 

8-11 7C63D350 A34C4EA6 010595F0 00FDBF00 

12-15 00000000 7C63C028 01A208B6 00000000 

NDSP..... 00000000 JSCB..... 009FC07C 

PRB: 009DA100 LINK..... 009DA188 

WLIC..... 00020023 OPSW..... 070C0000 813A8D42 

GPR0-3... 00000000 7F6F1378 00000048 82559B70 

GPR4-7... 2379EF08 2379EF30 02559C10 2375E140 

GPR8-11.. 7F6F1378 7F6F1378 2375E138 2375E31F 

GPR12-15. 2375D320 00007B38 00000001 7F6F1378 

EP....... BPXFSLIT MAJOR.... BPXINPV2 ENTPT.... A37807D8 

12 



Back to the ANALYZE report ... 




REQUEST = 11/15/2011 17:36:15.144127 


DATA=EXCLUSIVE RETADDR=A36A6452 

REQUEST = 11/15/2011 17:46:00.160780 



REQUEST = 11/15/2011 19:44:17.379913 

13 



The (almost) complete situation 

BPXFSMNT 

wants 

File System 

held by 

TCB 9A1050 

to mount 

*AMD/u$/tu 

Mount Latch 

and posts now 

BPXFSLIT to 

get response 

require 

and enters 

a wait. 

BPXFSLIT 

TCB 9DA188 

 

14 



What is BPXFSLIT doing 

• the WAIT bit in the LINK is off – task is *NOT* in a WAIT 

• WLIC 23 – WTO appears to be residual because 

• OPSW points to IEAVEPS1 – pause the currently executing task 

• Linkage Stack showed a PC to ISGLRTR in GRS address space 

• obtain the mount latch 

15 



What is BPXFSLIT doing (cont.) 

IP SUMMARY FORMAT ASID(16);f ‘ tcb: 009DA188‘ 

LINKAGE STACK ENTRY 01 LSED: 7F41E138 

LSE: 7F41E018 

GENERAL PURPOSE REGISTER VALUES 

00-01.... 00000000 00000000 00000048 7F6F1378 

02-03.... 00000000 00000048 00000000 82559B70 

04-05.... 00000000 2379EF08 00000000 2379EF30 

06-07.... 00000000 02559C10 00000000 2375E140 

08-09.... 00000000 7F6F1378 00000000 7F6F1378 

10-11.... 00000000 2375E138 00000000 2375E31F 

12-13.... 00000000 2375D320 00000000 00007B38 

14-15.... 00000000 80FDAD88 00000000 A37807D8 

... 

PSW...... 07040000 80000000 PSWE..... 00000000 00FDAD88 

TARG..... 00000000 A378080C MSTA..... 24F30000 00000000 

16 




IP CBF 24F30000 STR(BPXSTACK) ASID(16) 

Stack Entry 0 

Stack Entry Address: 24F30200 

Previous Entry Address: 00000000 

Next Entry Address: 24F31210 

Entry Point ID: 0503 

Csect: BPXFSLIT at 237807D8 

Entry Point: BPXFSLIT at 237807D8 

Stack Entry 1 * Active * 

Stack Entry Address: 24F31210 

Previous Entry Address: 24F30200 

Next Entry Address: 24F32EF8 

Entry Point ID: 1288 

Csect: BPXTXRIN at 23793468 

Entry Point: BPXTXRIN at 23793468 

17 




• BPXFSLIT called BPXTXRIN 

• BPXTXRIN is called by initialization, re-init and SETOMVS processing 

• as per ANALYZE RESOURCE: 

RESOURCE #0133 IS HELD BY: 



REQUEST = 11/15/2011 17:36:15.057144 

GRANT = 11/15/2011 17:36:15.142437 




REQUEST = 11/15/2011 17:36:15.144127 

18 




• the SYSLOG shows: 

MVSS 11319 18:36:14.88 

SETOMVS FILESYS,FILESYSTEM='OMVS.CPXMVS.DFS',SYSNAME=MVSS 

• DEADLOCK is between the MOUNT task and the SETOMVS task 

• MOUNT task holds MOUNT LATCH and posts SETOMVS task due 

to SECLABEL being set 

• SETOMVS task is itself waiting for the MOUNT LATCH due to 

SETOMVS FILESYS command 

• both tasks cannot proceed – DEADLOCK! 

19 



OA38094 – deadlock on mount latch 

BPXFSMNT 

wants 

File System 

held by 

TCB 9A1050 

to mount 

*AMD/u$/tu 

Mount Latch 

and posts now 

BPXFSLIT to 

get response 

require 

and enters 

a wait. 

BPXFSLIT 

TCB 9DA188 

SETOMVS 

20 



The Dining Philosophers Solution 

Quelle: Wikipedia 

21

OMVS latch deadlock –

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?