The Mainframe and Virtualization - LRR

IBM System z 

The Mainframe and Virtualization 

Michael Störchle 

System z IT Specialist 

Stoerchle@de.ibm.com 

v 

It’s Business that Matters 

© 2007 IBM Corporation

IBM System z 

Trademarks 

It‘s Business that Matters 

The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: AS/400, 

DB2, e-business logo, ESCON, eServer, FICON, IBM, IBM Logo, iSeries, MVS, OS/390, pSeries, RS/6000, S/390, System z9, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM. 

The following are trademarks or registered trademarks of other companies 

Java and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countries. 

LINUX is a registered trademark of Linux Torvalds in the United States and other countries. 

UNIX is a registered trademark of The Open Group in the United States and other countries. 

Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. 

SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC. 

Intel is a registered trademark of Intel Corporation. 

* All other products may be trademarks or registered trademarks of their respective companies. 

NOTES: 

Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary 

depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that 

an individual user will achieve throughput improvements equivalent to the performance ratios stated here. 

IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply. 

All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental 

costs and performance characteristics will vary depending on individual customer configurations and conditions. 

This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. 

Consult your local IBM business contact for information on the product or services available in your area. 

All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. 

Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any 

other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. 

Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography. 

References in this document to IBM products or services do not imply that IBM intends to make them available in every country. 

Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use. 

The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may 

make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. 

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the 

materials for this IBM product and use of those Web sites is at your own risk. 

2 IBM System z 


IBM System z 

Agenda 


1 Server Virtualization Basics and Business Value 

2 IBM System z Virtualization Differentiation 



IBM System z 


Virtualization – Say What? 

Virtual Resources 

� Proxies for real resources: same interfaces/functions, different attributes. 

� May be part of a physical resource or multiple physical resources. 

Virtualization 

� Creates virtual resources and "maps" them to real resources. 

� Primarily accomplished with software and/or firmware. 

Resources 

� Components with architected interfaces/functions. 

� May be centralized or distributed. Usually physical. 

� Examples: CPUs, memory, storage, network devices. 

� Separates presentation of resources to users from actual resources 

� Aggregates pools of resources for allocation to users as virtual resources 

4 

IBM System z 


IBM System z 


Virtualization – Say What? 

� Virtualization comprises the abstraction of physical systems to 

virtual systems 

– Divison of static relationships between logical system environments 

(environment of services and applications) and physical systems 

– Two possible directions: 

Integration of many single physical systems to one logical system 

Segmentation of one physical system into many logical systems 

– Such „logical systems“ are named as virtual machines 

– A virtual machine is a fully protected and isolated simulation of the 

underlaying hardware 

...altough virtualization comprises both integration and 

segmentation, this presentation concentrates on segmentation. 



IBM System z 


Important Terms Concerning Virtualization 

� Supervisor and Hypervisor 

– Supervisor is another term for an operating system of an virtual machine 

Controls the virtual machine and its dedicated resources 

– Hypervisor (or Virtual Machine Monitor) is another term for an controller of 

virtual machines 

Controls the physical system resources and dedicates them to virtual machines. 

Controls and handles processes of virtual machines which are critical to physical hardware 

Isolation of virtual machines 

Switching (context switching) between virtual machines (e.g., exits, time slicing...) 

Supervisor Supervisor Supervisor Supervisor 

Hypervisor 

Physical Resources 



IBM System z 



� Kernel (Privileged) Mode and User Mode 

– Kernel Mode provides full access to system resources. It is the mode of the 

operating system which administers and dedicates physical system resources. 

– User Mode provides restricted access to system resources (e.g., applications) 

� Privileged and Non-Privileged instructions 

– Privileged instructions can only be executed within Kernel Mode 

� Sensitive and Non-Sensitive instructions 

– Sensitive instructions invoke critical hardware areas 

Application 

Operating 

System 

Non-Privileged 

Privileged 

Application 

Operating 

System 

Hypervisor 

Non-Privilege 

Non-Privileged 

Privileged 



IBM System z 



� What is a Virtualizable Architecture? 

– Sensitive Instructions are also Privileged 

– E.g., System z 

� What is a Non-Virtualizable Architectur? 

– Sensitive Instructions are NOT always also Privileged 

– E.g., x86 



IBM System z 


Server Virtualization Approaches 

Hardware Partitioning Bare Metal Hypervisor Hosted Hypervisor 

Partition 

Controller 

Apps 

OS 

... 

Adjustable 

partitions 

SMP Server 

Apps 

OS 

Server is subdivided into fractions 

each of which can run an OS 

Physical partitioning 

S/370 SI-to-PP and PP-to-SI, 

Sun Domains, HP nPars 

Logical partitioning 

pSeries LPAR, HP (PA) vPars 

Apps 

OS 

... 

Hypervisor 

SMP Server 

Hypervisor software/firmware 

runs directly on server 

System z PR/SM and z/VM 

POWER Hypervisor 

VMware ESX Server 

Xen Hypervisor 

Hypervisor software runs on 

a host operating system 

VMware GSX 

Microsoft Virtual Server 

HP Integrity VM 

User Mode Linux 


© 2007 IBM Corporation 

Apps 

OS 

Type 1 

Hypervisor provides fine-grained 

timesharing of all resources 

• Hardware partitioning subdivides a server into fractions, each of which can run an OS 

• Hypervisors use a thin layer of code to achieve fine-grained, dynamic resource sharing 

• Type 1 hypervisors with high efficiency and availability will become dominant for servers 

• Type 2 hypervisors will be mainly for clients where host OS integration is desirable 

Apps 

OS 

... 

Hypervisor 

Host OS 

SMP Server 

Apps 

OS 

Type 2 

Hypervisor uses OS services to 

do timesharing of all resources

IBM System z 

Virt Mach 

L 

A 

ST 

PrivOp 

L 

... 


Trap and Emulate 

Examples: CP-67, VM/370 

Benefits: Runs unmodified OS 

Issues: Substantial overhead 

Hypervisor Calls (“Paravirtualization”) 

Virt Mach 

L 

A 

ST 

Hcall 

L 

... 

Trap 

Call 

• VM runs in user mode 

• All privileged instructions 

cause traps 

Hypervisor PrivOp 

emulation code 

• VM runs in normal modes 

• OS in VM calls hypervisor 

in case of critical Syscalls 

Hypervisor 

service 

Examples: POWER Hypervisor, Xen (today), 

HP Integrity VM 

Benefits: High efficiency depending of 

Hypervisor code + eventual HW support 

Issues: OS-Kernel must be modified to issue Hcalls. 

OS & Hypervisor levels must be in sync 

Hypervisor Implementation Methods 

Translate, Trap, and Emulate 

Virt Mach 

L 

A 

ST 

TrapOp 

L 

... 

Direct Hardware Virtualization 

Virt Mach 

L 

A 

ST 

PrivOp(*) 

L 

... 

• VM runs in normal modes 

• HW does most of the virtualization 

• SIE arch – set architecture of VM, 

provide status, translation & assists) 

• Hypervisor provides control 

Hypervisor 

service 

10 

IBM System z 


Trap 

Exit 

• VM runs in user mode 

• Some IA-32 instructions must 

be replaced with trap ops 

Hypervisor PrivOp 

emulation code 

Examples: VMware (today), Microsoft VS 

Benefits: Runs unmodified, translated OS 

Issues: May have some substantial overhead 

(*) ONLY for some 

control instructions, 

executed rather 

infrequent 

Examples: PR/SM, z/VM (also use hypervisor 

calls for a some functional enhancements) 

Benefits: Highest efficiency depending on 

HW/ucode support. Runs unmodified OS 

Issues: Requires HW & ucode support

IBM System z 


Partitioning Defined 

� Partitioning is the division of a single server’s resources* into multiple, independent, 

isolated systems capable of running their own operating system 

� Three types of partitioning: 

– Hardware – resources are allocated to partitions on a one-to-one basis with the 

underlying physical hardware (no sharing among partitions) 

– Logical – resources are managed by hardware firmware and allocated to 

partitions with a finer granularity than hardware partitioning (resource sharing 

among partitions) 

– Software – resources are managed by a software layer, aggregated into shared 

resource pools, and apportioned to users as virtual system resources, separating 

the presentation of the resources from the actual physical entities 

* Resources include: processors, memory, I/O adapters and devices, networking interfaces, co-processors 



IBM System z 


Hypervisor Technologies 

“Trap and Emulate” method 

� Guest OS runs in user mode 

� Hypervisor runs in privileged mode 

� Privileged instructions issued by guest operating system(s) are trapped by 

hypervisor 

� IA-32 (Intel) complications: 

Some instructions behave differently in privileged and user modes 

User mode instructions that access privileged resources/state cannot be trapped; 

instruction must be changed to something that can be trapped 

� Some guest kernel binary translation may be required - “Translate, Trap and 

Emulate” method 

Reduces resource-overhead 

� Originally used by mainframes in 1960s and 1970s (VM/370) 

� Used today by VMware 



IBM System z 



Hypervisor Call method (“Paravirtualization”) 

� Guest OS runs in privileged mode 

� Hypervisor runs in super-privileged mode 

� Guest OS kernel (e.g., AIX, i5/OS, Linux) is modified to do hypervisor calls for I/O, 

memory management, yield rest of time slice, etc. 

� Memory mapping architecture is used to isolate guests from each other and to 

protect the hypervisor 

� Used by POWER5 today 



IBM System z 



Direct Hardware method 

� Guest OS runs in privileged mode 

� Guest OS can be run unmodified, but can issue some hypervisor calls to improve 

performance or capability 

I/O (z/VM) 

Yield time slice (PR/SM TM and z/VM) 

� Extensive hardware assists for hypervisor (virtual processor dispatching, I/O 

pass-through, memory partitioning, etc.) 

� Used by System z (PR/SM TM and z/VM) 



IBM System z 


Let’s talk about… 

Business Value! 



IBM System z 


What are data centres facing today? 

Exploiding numbers of systems! 



IBM System z 


Data Center Heating 

Source: Uptime Institute, 

Footprint – Heat Density Trends 



IBM System z 


Why perform the effort of virtualization? - The Business Value! 

� Reduced hardware costs 

– Higher physical resource utilization 

– Smaller footprints 

� Reduced management costs 

– Fewer physical servers to manage 

– Many common management tasks become much easier 

� Improved flexibility and responsiveness 

– Virtual resources can be adjusted dynamically to meet new or changing needs 

and to optimize service level achievement 

– Provisioning and removing of servers within minutes 

Roles: 

Consolidations 

Dynamic provisioning / hosting 

Workload management 

Workload isolation 

Software release migration 

Mixed production and test 

Mixed OS types/releases 

Reconfigurable clusters 

Low-cost backup servers 

Virtual Servers 


Virtualization based on SHARING RESSOURCES 

allows an installation to grow 

dynamically both Vertically & Horizontally 

on the same server 

assuming we are dealing with 

Efficient Virtualization 

Physical 

Server 

Benefits: 

Higher resource utilization 

Greater usage flexibility 

Improved workload QoS 

Higher availability / security 

Lower cost of availability 

Lower management costs 

Improved interoperability 

Legacy compatibility 

Investment protection 



IBM System z 

Agenda 


1 Server Virtualization Basics and Business Value 

2 IBM System z Virtualization Differentiation 



IBM System z 


IBM Virtualization Evolution 

40 Jahre kontinuierliche Innovation 

Der IBM Mainframe ist Pionier und Perfektionist auf dem Gebiet der Virtualisierung 

System i & p haben fortschrittliche Hypervisor mit PR/SM-ähnlicher Funktionalität 

Andere Server hinken mit ihren Virtualisierungsmechanismen nach 

S/360 

CP-67 

Programmable 

Operator 

Hypervisor Control 

Program 

Z900 / z890 

1967 1970 1980 1990 2000 

Mainframe 

Z990 / z890 

Z9 EC/BC 

zAAPs 

ZIIPs 



zVM 

zAAPs 

ZIIPs 

VM/370 

VM/ESA 

VM/XA 

VM/HPO 

VM/SP 

® 

EXEC 

9672 G2 - G6 

z/VM 

9x21 

3090 

64-bit 

308x 

303x 

PR/SM 

QDIO Adapter 

ESA 

4381 

Interrupt Assist 

158/168 

EMIF FCP / SCSI 

64 MB real 31-bit 

Guest Lan 

Dynamic Stg 

SIE 

Reconfig II HiperSockets 

instruction 

(DSR II) 

N-way 

CMS pipelines 

Virtual Switch 

XEDIT 

Logical 

Virtual 

VM assist 

Partitioning 

Disk 

Integrated Facility 

microcode 

REXX TRACE 

For Linux (IFL) 

Multiple 

Subchannel 

Sets 

other 

assists 

Multiple 

Channel 

Sub- 

Systems 

PER 

IUCV TRAP 

IRD 

Advanced 

Paging 

Subsystem 

Resource 

Capping 

Performance 

Toolkit 

pAVE 

Micropartitioning 

Shared 

File 

System 

i & pSeries 

AS/400 

LPAR VIOS 

LPAR DLPAR 

Inte VT 

XEN AMD SVT 

VMware 

N-Port 


(NPVI) 

Unix 

x86 

LPAR 

Group 

Capacity 

2007

IBM System z 

Memory 


System z Virtualization Architecture 

HiperSockets & Virtual Networking and Switching 

VM 

VM 

Test 

Web 

Sphere 

VM 

SysAdmin 

Tools 

Apache 

Linux Linux CMS Linux 

SIE & SIE Assists 

z/VM z/OS z/OS z/VM 

IFL Processors 

LPAR 1 

VM 

I/O & Network 

WebSphere 

Memory 

LPAR 2 

Processors 

Traditional 

OLTP and 

Batch 

WLM WLM 

Memory 

LPAR 3 

HW/ucode functions - SIE & SIE assist 

Processor Resource / System Manager (PR/SM) 

VM VM 

FTP 

Linux 

Test 

Memory 

z/OS 

SIE & SIE Assists 

LPAR 4 

Intelligent Resource Director (IRD) 

Multi-Dimensionale Virtualisierung 

� HW: PR/SM (LPAR’s) & SW: zVM (VM’s) 

� Isolierte Umgebungen: EAL5 für PR/SM und 

EAL3+ für z/VM 

� Hochentwickelter Hypervisor 

� SIE Instruktion: Virtualsierung ist eingebaut, kein 

Add-On 

� 10% der Integrierten Schaltkreise werden für 

Virtualisierung benutzt (SIE) 

� Zeit- und Ereignisgesteuertes Dispatching 

� SHARED ALL Architektur 

� Jegliche virtuelle CPU kann auf jeder beliebigen 

physischen CPU betrieben werden. Sharing ist 

bis auf “1”%-Ebene möglich 

� Shared oder Dedizierter Pool an *CPUs” 

� Garantierte LPAR Kapazität 

� Physische & Virtuelle Ressourcen (CPU, I/O und 

Hauptspeicher) können dynamisch innerhalb und 

entlang der LPARs angepasst werden 

� LPAR Zoning: Jede LPAR hat eine 0-Origin. Das 

erlaubt I/O-Zugirffe auf den Hauptspeicher einer 

LPAR ohne Hypervisor Eingriff 

� z/VM kann virtuelle Devices erschaffen, die 

physisch nicht vorhanden sind 

� z/VM hat raffinierte Scheduling-Algorithmen um 

das Gesamtsystem für Reaktion und Throughput 

zu optimieren 



IBM System z 



SIE-Instruction! 



IBM System z 


System z Interpretive Execution – 

Advanced Technology for Virtual Server Hosting 

� Start Interpretive Execution (SIE) instruction 

– Establish the full architectural capabilities of an 

architecture for the guest 

– Supports MULTIPLE architectures CONCURRENTLY 

for multiple guests 

– Allows guests to run in NORMAL mode (no OS 

modifications needed/no TRAP’ing) 

– Reduces context switch time 

– Multiple control register sets 

– PR/SM AND z/VM exploit SIE 

No performance penalty for running z/VM in an LPAR 



IBM System z 


How: Start Interpretive Execution (SIE) 

� SIE = “Start Interpretive Execution”, an instruction 

� z/VM (like the LPAR hypervisor) uses the SIE instruction to “run” virtual 

processors for a given virtual machine. 

� SIE has access to: 

– A control block that describes the virtual processor state (registers, etc.) 

– The Dynamic Address Translation (DAT) tables for the virtual machine 

� z/VM gets control back from SIE for various reasons: 

– Page faults 

– I/O channel program translation 

– Privileged instructions (including CP system service calls) 

– CPU timer expiration (dispatch slice) 

– Other, including CP asking to get control for special cases 

� CP can also shoulder tap SIE from another processor to remove virtual 

processor from SIE (perhaps to reflect an interrupt) 



IBM System z 


Basic Direct HW virtualization – transparent to applications/OS 

Logical CPU Hypervisor 

Physical CPU 

Program 

Instruction 

stream 

Application 

Instructions 

High 

Frequency 

Control 

Instructions 

that require 

“emulation’ 

modifications 

Low 

Frequency 

Control 

Instructions 

that require 

emulation 

modifications 

Interception 

Guest leaves 

SIE 

Modify 

Subchannel 

instruction, 

etc. 

PRSM-LPAR 

or 

VM 

SIE 

Dispatch Guest 

(establish 

architecture) 

Load, Store, 

Add, Move, etc. 

Start Subchannel, Test 

Subchannel, etc. 

. 

SIE 

Re-dispatch 

guest 

PRSM-LPAR 

or 

VM 

Hypervisor 

HW 

Instruction 

Interpretation Controls 

SIE 

STATE 

Description 

Ucode 

Firmware 

High perform 

Instruction 

Interpretation 

Handling 

SIE 

Assists 

Instruction 

Execution Unit 

Instructions 



Load 

Store 

Add 

…... …........ 

SSCH 

TSCH 

MSCH 

…..... 

System z with SIE 

(Start Interpretive Instruction Execution) 

* System z runs ALWAYS in PR/SM-LPAR 

mode under SIE 

* LPAR is the “only” game in town, 

meaning performance items and other 

functionalities is developed accordingly 

zVM invokes SIE to run VM’s 

(SIE under SIE) 

* Efficient for performance - and new 

version of OS and Hypervisor 

Positioning of System z & Intel/AMD 

� System z: 

the requirement for the underlying HW 

support is NOT an issue for System z, - since 

the basic System z Architecture & HW design 

has implemented this for “decades”. 

� Intel and AMD 

is developing some virtualization HW support 

based on a similar structure like the SIE 

architecture as it was externally documented 

25 years ago. 

The amount of HW and SIE assist 

functionalities are seemingly rather limited at 

this point. 

No “SIE” under “SIE

IBM System z 



Scheduling/Dispatching of VMs! 



IBM System z 


Classic Scheduler / Dispatcher Picture 

User goes idle 

Dispatch 

List 

VMDBK D1 

. 

. 

. 

. 

. 

VMDBK Dn 

User logs off 

Resources available 

(time slice and priority basis) 

Minor time slice expires 

other interrupt or intercept 

Elapsed time slice expires 

...ready 

...wait state 

Dormant list 

Eligible 

List 

VMDBK E1 



. 

. 

. 

. 

. 

VMDBK En 

User logs on 

User becomes 

runnable

IBM System z It‘s Business that Matters z/VM - Flow 

A Dispatcher 

B 

Dispatcher 

(if pending 

Interception 

Indicated) 

Run the virtual machine 

Open interruption window (enable/disable) I/O or external interruption handler 

Check for pending simulation work Validate guest PSW 

Queued guest interruptions 

Enter console-function mode 

Purge Machine Lookaside if scheduled 

Load guest state into registers 

Enable I/O, external interruptions 

SIE 

Interception 

- Disable interruptions 

- Save guest state 

Process interception from SIE depending 

in interception code 

- Instruction interception 

- Program interruption 

- I/O or external request 

- Guest wait state 



A 

B 

Reflect if enabled 

Intercept enablement if disabled 

Detect guest wait state 

Host interruption (disable on interruption) 

Program interruption 


- Resolve fault or Reflect 

exception to guest 

B 

- Simulate instruction 

- Simulate program interruption 

- Translate program interruption 

- Schedule interruption Scan 

C 

I/O or external Interruption 


- Check for pending interseption 

- Process interruption 

A 

C 

Guest EndOp 

- Reflect per event or 

exigent machine check 

- Mark guest EndOp 

B

IBM System z 



Configuration of VMs! 



IBM System z 


Configuration of LPARs via HMC 



IBM System z 


Configuration of VMs within z/VM 

� Is done by a Flat-File called „User Directory“, owned by a service 

VM called „Maint“ (maintenance) 

How to create a new VM? 

� Log-On to the service VM called „Maint“ 

� Open file user direct a 

� Configure the new VM...but how to do that? – Remeber the basic 

components! 



IBM System z 


Configuration of VMs within z/VM 

USER LINUX01 MYPASS 512M 1024M G 

MACHINE ESA 2 

IPL 190 PARM AUTOCR 

CONSOLE 01F 3270 A 

SPOOL 00C 2540 READER * 

SPOOL 00D 2540 PUNCH A 

SPOOL 00E 1403 A 

SPECIAL 500 QDIO 3 SYSTEM MYLAN 

LINK MAINT 190 190 RR 

LINK MAINT 19D 19D RR 

LINK MAINT 19E 19E RR 

MDISK 191 3390 012 001 ONEBIT MW 

MDISK 200 3390 050 100 TWOBIT MR 



IBM System z 



Performance! 



IBM System z 


Large Linux guest with z/VM 5.2 

Throughput 

2 4 6 8 10 12 16 24 48 

Large guests can now be run under z/VM without special 

treatment in the Linux DASD driver 

SLES9, z/VM 5.2 

IBM System z 

Informix database server 

Linux Memory (GB) 

LPAR z/VM 5.2 

guest 

z/VM 5.1 

guest 

Performance... 


IBM System z 


HTTP 

HTML 

GUIs 

SSL 

Web Servers SET 

Browsers 

TCP/IP 

Java 

Open Standards 

Questions? 

XML 

Linux 

Open Source 

Stoerchle@de.ibm.com 



IBM System z 

Hindi 

Traditional Chinese 

Tamil 


Grazie 

Italian 

Russian 

Arabic 

Thank You 

English 

Simplified Chinese 

Japanese 

Gracias 

Spanish 

Obrigado 

Brazilian Portuguese 

Merci 

Danke 



French 

Korean 

Thai 

German

The Mainframe and Virtualization - LRR

Create successful ePaper yourself

Delete template?

Save as template?