IBM Tivoli Workload Automation - Nordic TWS conference

IBM Software Group 

® 

IBM Tivoli Workload Automation 

Latest GA updates 

Flora Tramontano – Flora.Tramontano@it.ibm.com 

TWA Product Manager 

IBM Software Group 

© 2009 IBM Corporation

IBM Software Group | Tivoli software 

Tivoli Workload Automation - last updates timeline 

Current versions 2H07 

1H08 3Q08 4Q08 1Q09 2Q09 

Tivoli 

Workload 

Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 

Reporting feature 

Virtual Workstation 

IPv6 

NOERROR enhancements 

V8.3 SPEs 

Workload Service 

Assurance 

V8.3 SPEs 

Enhanced JCL Variables 

V8.5 

Conditional dependencies 

Enhanced event-driven 

management 

OPC removal 

TEP improvements 



Scheduler 

Tivoli Workload 

Scheduler for 

Applications 

V8.3 

V8.3 

V8.4 

Event based scheduling 

TEP integration 

IPv6 support 

LDAP support 

Consumability improvements 

V8.4 

V8.4 FP1 

CCMDB Integr. 

Events for business logic 

Eclipse-based TWS 

Workbench 

V8.4 FP1 

V8.5 

VarIable tables for streams 

Streamline configuration (simplify 

deployment scheme) 

Workload Service Assurance 

V8.5 


Dynamic 


Broker 

V1.2 

SAP load balancing 

SAP events in EDWA 

Import SAP calandars 

BDC session management 

IDOCs management 

Process chains 

Basic Job throttling 

Monitor Alerts 

Solution Manager integration 

and certification 


Dynamic 


Console 

V8.3 

V8.4 



LDAP and IPv6 support 

SAP job submission (alignment to JSC) 

z/OS 8.3 features alignment 

V8.4 FP1 


Assurance 

V8.5 

Modeling and Administration on 

Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

2




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 

V8.3 

TWS z/OS V8.3 

SPE 

TWS V8.3 V8.5 

V8.4 



IPv6 support 

LDAP support 


V8.4 

V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 

V8.5 

Variable tables for streams 

Installation improvements 

(deployment scheme) 


V8.5 


Dynamic 


Broker 

V1.2 













Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

3

No risk 

No risk 

No risk 

No risk 

No risk 

No risk 

No risk 

No risk 


Zooming : Workload Service Assurance 

Business needs 

How to align workload to business priorities? 

How to improve Service Level Agreements, 

automate operations and reduce costs? 

How to monitor and measure the impact of 

workloads against business service delivery? 

How can I filter and prioritize critical events 

from a flood of system-wide events? 



Roles and interfaces 

Tim: the administrator 

Dean: the operator 

Modeling critical workload 

Identifying critical jobs 

Preliminary settings for critical 

path calculation and automatic 

promotion 

Monitoring critical workload 

achievement 

Monitor how well critical workload 

is proceeding to milestone points 

Prompt proactive IBM Software human Group reactions | Tivoli software 

is needed 

TWS z/OS ISPF Panels 

Job Scheduling Console 

Graphical User Interface 

Workload Service Assurance: Monitoring phase 

The ultimate scenario 

1.The WEB UI critical jobs dashboard 

gives at a first glance the view of how 

well the critical workload is 

proceeding 

2.Pipe of high risk (red), potential risk 

(yellow) and no risk (green) jobs is 

provided 

3.It is likely that Dean explore high risk 

(red) job first 

The ultimate scenario 

Tivoli Dynamic Workload 

Console – Web-based UI 


Workload Service Assurance: Monitoring phase 

5 

2 

2 

6 

12 

How can I ensure business priorities to get the 

right attention, and speed their implementation 

time? 

10. Dean task ends when all critical jobs 

are on track! 

15 

How can I enable proactive reaction? 

4



Administration 

Automation 

Automation 

Operation 

Solution 

Allows identification of critical 

workload 

Calculates critical path to critical 

workload and keep it 

dynamically updated 

Takes autonomous remedial actions 

for lagging jobs in 

the critical path 

Gives a view of the critical end 

points and the possibility 

to understand how well workload is 

progressing to reach 

that point to trigger proactive 

human reaction 

Customer value 

Alignment of workload to 

business priorities 

Service Level management, 

automate operations and reduce 

costs 

Take in account historical 

metrics and indicators when 

forecasting expected completion 

of jobs 

Monitor the most current critical 

path 

Automate remedial actions as 

first reaction to risk 

Proactive alerting of users about 

potentially risk conditions 

Forward information to TEP and 

TBSM 

5


Workload Service Assurance: Critical Path determination 

Automatic determination of critical path to critical jobs 

Through the network of internal and external dependencies 

Path with minimum slack time 

Any delay affects the critical job 

Dynamic adjustements: an alternate leg may become critical 

Unexpected anomalous condition in the critical jobs predecessors 

Unplanned workload 

21:00 5:00 

1:00 

6:00 

6


Workload Service Assurance: Promotion on Critical Path 

Jobs lying on the Critical Path are automatically promoted if they are late to 

turn the workload back on track 

Jobs starting late are given highest priority for fast submission 

Jobs running late are promoted to a better performance Workload Manager Service 

Class! 

21:00 5:00 

1:00 

6:00 

7


Workload Service Assurance: Monitoring critical jobs 

An easy and effective way to monitor critical jobs is through the Tivoli Dynamic 

Workload Console dashboard. 

Critical job estimated end is 

beyond its deadline 

Critical job is on track but some 

predecessors, not necessarily 

on the critical path, are in a 

warning condition, for example 

in error or with the priority set to 

zero. 

Critical job is on track 

8


Workload Service Assurance: Monitoring critical jobs 

9




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 


Dynamic 


Broker 

V8.3 

V8.3 

V1.2 

V8.4 

V8.5 



IPv6 support 

LDAP support 


V8.4 



Enhanced SAP events event-driven 

in EDWA 


management 

OPC removal 


V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 




V8.5 





V8.5 






Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

10



Business scenario 

JOB1 

The energy company needs to automate 

conditional execution logic, and 

dynamically perform different branches of 

invoice processing for different typology of 

clients. 

RC=0 

JOB2 

RC=4 

RC0 and RC4 

The organization desires to flexibly 

implement self-recovery system, 

automating complex branches of recovery 

processing. 

JOB3 

JOB4 

JOB5 

General need is to support real-time 

processing decisions, based upon 

combined conditions on return codes and 

statuses of specific workloads. 

JOB6 

JOB3 status = C OR 


JOB6 status = C 

11



Solution 

Simple static workflow definitions (job 

execution conditioned on successful 

completion of predecessor jobs) are not 

enough to map dynamic and complex 

workflows. It will be possible to define 

combined workflows definitions on return 

codes/statuses. 


Easily map business processes with reduced TCO 

Replace IF/THEN/ELSE JCL coding to simulate branches of workflows 

Easy monitoring of workflows 

12



Operation level 

OP1 in ACONDB1 

RC LT 4 

RC GE 4 

Condition definition components 

OP1 in ACONDC1 

OP1 in ACONDC1 

Relational Operator (GE, GT, LE, LT, EQ, NE, RG) 

Condition Type (RC , STATUS) 

A1 

A2 

A3 

Rule (AND, OR) 

OR, AND, 

N out of M 

B 

13



Rule 

A1 

A2 

A3 

A1 

A2 

A3 

A1 

A2 

A3 

or 

or 

B 

or 

N out M 

B 

A4 

and 

and 

B 

and 

At least one of A1, A2 A3 

Cond deps must be true 

At least 2 out of 4 A1, A2, 

A3, A4 cond deps must 

be true 

All of A1, A2 A3 

cond deps must be true 

14


Condition example 

Start operation Z1 when the following Condition is true: 

((A1 RC > 0) and (A1 RC 0) and (C1 RC


Condition example 

Start operation Z1 when the following Condition is true: 

((A1 RC >= 4) and (B1 RC >= 4)) or ((C1 RC >= 8) and (D1 RC >= 8)) 

A1 B1 C1 

D1 

All must be true: 

A1 rc GE 4 

B1 rc GE 4 

and 

Y1 

and 

Y2 

All must be true: 

C1 rc GE 8 

D1 rc GE 8 

or 

Y1 and Y2 are 

DUMMY operations 

Z1 

At least one must be true: 

Y1 Status = C 

Y2 status = C 

16


Example - Autorecovery application 

APPLA is designed in a such a way to manage possible operations errors 

APPLA 

JOBA 

JOBA rc LT4 

JOBA rc GE4 

JOBRLT4 

JOBREGT4 

JOBB 

At least one must be true: 

JOBA rc=0 

JOBRLT4 rc=0 

JOBREGT4 rc=0 

17


Example – Autorecovery Application 

JOBA performs a critical task 

JOBRLT4 is the recovery job that handles error condition JOBA RC=4 

JOBB must be run after successfull completion of JOBA or successful recovery 

through JOBRLT4 or JOBREGT4 

18


Autorecovery Application- Definition 

JOBRL4 - Define conditions 

JOBRL4 - Define condition dependency (Run if 0 < JOBA


Autorecovery Application - Definition 

JOBB - Define condition dependency (Run if JOBA=0 or JOBRL4=0 or JOBRGE4=0) 

20



Define condition dependency through Tivoli Dynamic Workload Console 

21



Define condition dependency through Tivoli Dynamic Workload Console 

22


How TWSz evaluates conditions 

UNDECIDED FALSE 

JOBA 

RE 

UNDECIDED TRUE 

 

APPLA instance is ran 

WX 

JOBRL4 

0 < RC


How TWSz evaluates conditions 

Conditional Dependency status / evaluation 

T True 

F False 

U Undecided 

Evaluation is done each time an involved operation status changes 

Until final values: True, False 

Operations status 

An operation with condition dependencies defined changes status among: 

W Waiting - Conditional dependency is not yet valuable 

R Ready - Conditional dependency is True 

X X (dummy complete) - Conditional dependency is False 

 

 

X status is propagated to any normal successor at any level 

To prevent that, you can define a condition dependency based on the X 

status 

24


X status 

JOB1 

Propagation of X status 

RC=0 

RC0 and RC4 

Unless you define a condition 

dependency on X status 

X 

JOB2 

RC=4 

JOB4 

JOB5 

Ex: JOB3 cond dep on JOB2 ST X 

X 

JOB3 

X 

C 



JOB6 status = C 

JOB6 

C 

25


Meaning of recovery operation 

 

In the previous example, APPLA instance is in Error 

status and therefore it is not deleted at Daily Plan 

Operation 

APPLA 

Status 

JOBA 

E 

 

Nevertheless, you may want to consider the instance 

successfully completed, since you designed the job 

JOBREGT4 to recover JOBA error 

JOBRLT4 

JOBREGT4 

JOBB 

X 

C 

C 

 

In this case, when defining JOBRLT4 

and JOBREGT4, besides specifying 

JOBA as conditional predecessor, you 

can also set the COND RECOVERY 

JOB option to Y 

 

So, APPLA instance will be removed by 

Daily Plan 

26

Summary 


Defined in the AD operations 

Tracked in the Current Plan, either created at plan creation time or 

dynamically added 

Checked by Scheduler to decide if an operation is eligible to be 

submitted 

Condition dependencies can be modified in the Current Plan, if their 

status is Undecided 

27




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 


Dynamic 


Broker 

V8.3 

V8.3 

V1.2 

V8.4 



IPv6 support 

LDAP support 


V8.4 




V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.5 

V8.4 FP1 



management 

OPC removal 





V8.5 





V8.5 






Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

28


Event-driven workload automation 

Business scenario 

The pharmaceutical company needs to produce 

consolidated daily reports, gathering data 

provided with files FTP-ed on distributed 

systems 

General need is to raise the automation level 

one step, intercepting and reacting to a wider 

pool of events, and integrating to e-business 

applications - while simplifying existing 

mechanism 

Solution 

Easy and consolidate way to define flow of 

triggering events 

File dependencies on USS, Windows and Unix 

files 

29


Dataset triggering 

Besides the acquired dataset triggering method to define dataset triggering, now 

you have an additional easier and more powerful way to do it 

Acquired method 

Edit and compile assembler macros to 

build the triggering dataset table 

Manually deploy the table to each 

system where data set monitoring 

must be active 

New method 

Single centralized job to build and 

automatically deploy configuration files 

Define event rules, triggering files 

and destinations editing an XML 

sample 

Deploy configuration files to all the 

defined tracker destinations 

30


Customize triggering criteria, build and deploy config files 

31

Filewatcher 


Filewatch utility enables reactions to file system changes of 

files and directories 

HFS 

Windows and AIX files 

You can implement file dependencies defining a job that runs 

the filewatch utility 

Filewatch 

The Filewatch runs until the specified deadline or until the condition is matched 

Condition – File created, modification running, modification completed, modification 

running completed, deleted 

Condition is checked through file size and modification time changes, which are checked at 

user specified interval time. 

32

Filewatcher 


33




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 


Dynamic 


Broker 

V8.3 

V8.3 

V1.2 

V8.4 



IPv6 support V8.5 

LDAP support 


V8.4 

Variable SAP load tables balancing for streams 


Installation Import SAP improvements 

calandars 

(simplify deployment scheme) 


V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 




V8.5 





V8.5 






Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

34


Zooming : Variable tables 

Business scenarios 

To bring automation one step further, the news 

agency must specialize the treatment of wire 

news on each agent, but keep the flows global. 

This provides the lowest maintenance costs. 

Workstation class 

VT1 


Use of variable tables allows reducing the 

number of scheduling objects required to 

implement the desired workload ! 

Var2 

JS 

Var1 

VT2 

VT4 

var1=tws1 

VT3 

var1=twsalt 

Variables are often associated together. The 

administrator creates tables that are part of the 

best practices, so operators can share them and 

verify the tunings. 

Variables support the ease of transition between 

environments, with the same definition serving in 

two different environments. 

35

Variable tables 


New capability to define variables and variable tables 

The same variable may appear in different tables with different values 

It is possible to (optionally) assign a variable table to every run cycle, job stream and 

workstation 

Users can change the behavior of the workload according to when, why and where the 

workload is scheduled to run 

Using variable tables with Run Cycles (when) 

jobs and job streams scheduled in different days may 

behave differently 

Using variable tables with Job Streams (why) 

jobs and job streams created in different workflows may 


Using variable tables with Workstations (where) 

jobs and job streams running on different machines may 


Using default variable table for common behaviors and 

setting 

Resolution 

order: 

The variable will be 

substituted with the 

first value found for 

that variable 

36


Example – Using Variable Tables with Run Cycles 

Jack runs a weekly 

job, that parses and 

processes data. The 

job must process 

different kind of data 

once a month. 

• Creates a job that 

processes data 

contained in a 

variable path 

REP_PATH 

• Defines two 

variables tables with 

two different values 

for REP_PATH 

37


Example – Using Variable Tables with Run Cycles 

Jack creates for this 

• 

job 

Defines 

stream 

a weekly 

two run 

and 

cycles 

a monthly 

(monthly 

run 

and 

cycle 

weekly). 

for the 

Each 

job 

run cycle is assigned 

• Associates 

to the proper 

the two 

variable 

variable 

tables 

table. 

to the 

weekly and monthly 

run cycles 

38


Dynamic Business Process progress in TWS 8.5 

Business scenarios 

Better Business Process mapping means clearer 

definitions, less workarounds, and a more robust workload. 

Business 

event 

Param 

Param 

Param 

Rule 

Engine 

TWS 8.5 contribution 

Variable tables + Jobstream definition has flexibility and 

ease of maintenance 

Process definition can execute anywhere and receive 

parameters at submission, they can also pass parameters. 

JS 

(Process integration) 

A new TDWB workstation type describes better how to 

take advantage of the power of Tivoli Dynamic Workload 

Broker. 

Param 

Param 

39




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 


Dynamic 


Broker 

V8.3 

V8.3 

V1.2 

V8.4 




LDAP support 


V8.4 



Installation SAP events in improvements 

EDWA 


(simplify deployment scheme) 


V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 




V8.5 





V8.5 






Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

40



New installation launchpad 

Easy and guided installation of 

components 

Optimized disk space and 

memory consumption 

Simplified problem 

determination and 

troubleshooting 

Easy access to product 

documentations and samples 

41


Common Launchpad 

Easy and optimized installation 

TWS and TDWC installation are 

capable of reciprocal discovery 

If one of them is found you are given 

the option of installing them on the 

same shared WAS 

Log and trace files located in the 

same directory 

Automatic detection of specific failing 

points during installation 

Response file generated with the 

selections made, to replicate 

installation on other systems 

Documentation at fingertips 

Downloadable documentation 

IBM Tivoli Information Center 

IBM Publications Center 

Graphical representation of typical 

network topology layout for 

small/medium accounts 

42


Enhancements on the TSM-TWS integration agent 

TWS makes TSM actions more manageable 

TWS monitors and reacts on exception on large scale TSM workloads 

TWS reports on job execution and success statistics. 

TWS definition represents a portable, complete description 

TSM administrator can import, export, bulk-process, promote to prod the various 

definitions. 

TWS could provide numerous out-of-the-box templates of TSM actions. 

TWS orchestrates several TSM servers 

The dependencies available on TWS definitions can map constraints between 

TSM servers, like bandwidth, robot tape heads 

TWS has recovery built-in 

Simple or complex recovery actions can be triggered upon failure to implement a 

first reaction layer without human intervention. 

The Event-Based Workload Automation since TWS 8.4 can implement automatic 

escalation rules based on precise conditions. 

43


Equivalence of TSM and TWS command control 

Synchronous 

Asynchronous 

Client 

Admin 

TSM: dsmadmc 

TWS: CLIENTACT 

TSM: dsmadmc 

TWS: ADMIN 

TSM: Incremental, 

Selective, … etc. via 

DEFINE SCHEDULE 

TWS: CLIENTSCHED 

TSM: DEFINE SCHEDULE 

administrative version 

TWS: ADMIN 

44




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 


Dynamic 


Broker 

V8.3 

V8.3 

V1.2 

V8.4 




LDAP support 


V8.4 



Monitor SAP events Alerts in EDWA 




V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 




V8.5 





V8.5 






Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

45


Tivoli Workload Scheduler for Apps v8.5 

SAP Job Throttling 

Extensive use of the interception feature to ‘freeze’ SAP jobs and ‘release’ them to 

optimize the load of the SAP work processes. 

Solution Manager Certification and Integration 

Ability to send data about the job throttling component activities to the SAP CCMS 

architecture. This enables the job throttling to be included in SAP Solution Manager 

Business Management, which is based on the CCMS. 

Support SAP Monitoring Alerts 

SAP CCMS Alert Monitor generates alerts in the monitored system using threshold 

values and rules, as soon as the status of the monitored system deviates from the 

norm. TWS4APPS will take advantage of the SAP CCMS Monitoring Architecture by 

using the SAP BC-XAL API, the external interface for alert management. With this 

new functionality, a TWS event could be raised when CCMS Alert Monitor generates 

an alert. This feature extends the current EDWA infrastructure and can be used only 

in a TWSd environment. 

46




1H08 3Q08 4Q08 1Q09 2Q09 



Scheduler 

for z/OS 

V8.3 

V8.3 SPEs 



IPv6 


V8.3 SPEs 


Assurance 

V8.3 SPEs 


V8.5 



management 

OPC removal 




Scheduler 


Scheduler for 

Applications 

V8.3 

V8.3 

V8.4 



IPv6 support 

LDAP support 


V8.4 

V8.4 FP1 

CCMDB Integr. 



Workbench 

V8.4 FP1 

V8.5 

VarIable tables for streams 

Streamline configuration (simplify 

deployment scheme) 


V8.5 


Dynamic 


Broker 

V1.2 












Dynamic 


Console 

V8.3 

V8.4 






V8.4 FP1 


Assurance 

V8.5 


Single GUI 

V8.5 FP1 

Conditional 

dependencies 

alignment 

47

IBM Tivoli Workload Automation - Nordic TWS conference

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?