The Virtualization Cookbook - z/VM - IBM

IBM Systems and Technology GroupLPALPARs A02 and A2E: z/VM 6.3IDENTITY MAINT: z/VM system administrationIDENTITY TCPMAINT: TCP/IP administrationLPAOverview ofentire systemR1IDENTITY TCPIP: TCP/IP stackIDENTITY AUTOLOG1: z/VM configuration at IPLIDENTITY DTCVSW1/DTCVSW2: VSWITCH controllersUSER LNXMAINT: CMS files common to Linux systemsIDENTITY LNXADMIN: Linux system administrationUSER RH64GOLD - RHEL 6.4 golden imageRnUSER S112GOLD: SLES 11 SP3 golden imageUSER LINUX1: Linux virtual server 1...USER LINUX6: Linux virtual server 6OSA ExpressOSA ExpressPhysical switchPC LinuxNFS serverDesktopmachine© 2012 IBM Corporation4

System overviewIBM Systems and Technology Group© 2012 IBM Corporation5

IBM Systems and Technology GroupHistory of cookbooks6,7 The Virtualization Cookbook(s)for RHEL 5 and SLES 10, 3/075 The Virtualization Cookbook 2 1 , 8/062 The Virtualization Cookbook, 2/06Project started: 11/042004 2005200620073 Redbook: The VirtualizationCookbook for SLES9,SG24-6695-01, 4/061 Redbook published FromLPAR to Virtual Servers inTwo Days, SG24-6695-00: 6/054 Redbook: The VirtualizationCookbook for RHEL4,SG24-7272-00, 9/061includes middleware cloning© 2012 IBM Corporation6

IBM Systems and Technology GroupHistory of books (cont'd)See: http://www.vm.ibm.com/devpages/mikemac/9 The Virtualization Cookbookfor SLES 11, 2/102008 200920108 Redbook: The VirtualizationCookbook for SLES 10 SP2 2 , 10/08201110 Redbook: The VirtualizationCookbook for SLES 11 SP1, 1/112includes "travelling /home"11 Redbook: The VirtualizationCookbook for RHEL 6, 2/11© 2012 IBM Corporation7

IBM Systems and Technology GroupHistory of books (cont'd)See: http://www.vm.ibm.com/devpages/mikemac/12 The Cloud Computing Cookbook for z/VM6.2, RHEL 6.2 and SLES 11 SP2, 1/1213 The Virtualization Cookbook for z/VM6.2, RHEL 6.2 and SLES 11 SP2, 7/122012 20132014 ???14 Redbook: The VirtualizationCookbook for z/VM 6.?, RHEL 6.4and SLES 11 SP3, 8/13 (???)© 2012 IBM Corporation8

IBM Systems and Technology Group"Parts" in current book1. Introduction and z/VMa. Introduces z/VM 6.?b. Discusses planningc. Installation and configuration into a two member SSId. Service2. RHEL 6.4 Linux - install, customizing and clone RHEL3. SLES 11 SP3 Linux - install, customizing and clone SLES4. Other topics - includes chapters on:a. Live Guest Relocation (LGR) between SSI membersb. Configuring DirMaint, SMAPI and RACFc. Monitoring z/VM and Linuxd. Miscellaneous “recipes”5. Appendices - includes references, cheat sheets and lists the source code© 2012 IBM Corporation9

IBM Systems and Technology GroupChanges in the planned bookz/VM and Linux on IBM System z: The Virtualization Cookbook for z/VM 6.3 RHEL 6.4 andSLES 11 SP3 has many new sections:z/VM sections are updated for 6.3 with a two member SSI setupLinux sections are updated for both RHEL 6.4 and SLES 11 SP3Layer 2 virtual switches onlyDescription of "a private interconnect" (???)Installation of Non-SSI z/VM 6.? systemMultipathing of FCP/SCSI disksAddress z/VM HYPERPAVAutoYast on SLESDescribe VIR2REAL EXECSection on Kiwi on SLESBeef up "z/VM Live Guest Relocation" chapterAdd section on z/VM "LOGON BY" for audit trailsSection on the "Linux Terminal Server"Add a CRON Service Virtual Machine for z/VMDefine command-specific z/VM privilege classRed Hat Sattelite serverSLES "Live CD" installation (???)Example of setting up hipersockets to z/OSDescribe smaclient© 2012 IBM Corporation10

IBM Systems and Technology GroupChanges in the July 17, 2012 bookz/VM and Linux on IBM System z: The Virtualization Cookbook for z/VM 6.2 RHEL 6.2 andSLES 11 SP2 has many new sections:Title prefix is back.Steps for installing RACF into an z/VM 6.2 SSI cluster have been added.This configuration describes adding the UseRACF=yes setting to DirMaint.z/VM development now recommends the use of layer 2 virtual switches (VSWITCH)exclusively.How to attach z/VM TCP/IP stack to HA virtual switch.MAINT's slightly modified PROFILE XEDIT is now copied to the MAINT 19E disk so that itneed not be copied to many virtual machines 191 disk.Service section updated for z/VM 6.2 (now that the first RSU is available).An update to the CPFORMAT EXEC code has been made available. In the January 2012version of the code, while in a non-SSI environment, OWNER data was still being written toCP-owned volumes. That issue has been corrected.© 2012 IBM Corporation11

IBM Systems and Technology GroupIntroduction - Planning - bill of materialsHardwareSystem z LPARs (2 or 4 for SSI)IFLsMemory (aka storage)DASD (aka storage :))Two OSA cards for HA VSWITCH (One is OK)Temporary Distributed serverSoftwarez/VM 6.3LinuxSLES-11 SP3RHEL 6.4Code associated with book: http://www.vm.ibm.com/devpages/mikemac/SG248147.tgzNetworking resourcesTCP/IP address for each z/VM SSI membersTCP/IP address for each LinuxDNS names© 2012 IBM Corporation12

IBM Systems and Technology GroupIntroduction - Planning (cont'd)ConventionsVolume labeling conventionAddressVolume labels are only 6 charsDASD type - Minidisk or PERM spaceUsing device address in last 4 chars:LPAR identifierGuarantees unique labelsFirst character is LPAR identifierSecond character is function (P=page, S=spool, M=minidisk)File naming conventionFile that is shipped with VM/Linux - ORIG or .orig suffixFile that was last working - WRKS or .worksPassword convention - z/VM admin, Linux admin, Linux usersWorksheets - 2 sets of 4 worksheetsPopulated set of worksheets for examples used in the bookBlank set of worksheets forz/VM resourcesLinux resourcesz/VM DASDLinux virtual machinesMM E34A© 2012 IBM Corporation13

IBM Systems and Technology GroupIntroduction - Configure a desktop machineSSH clientPuTTY is describedSet SSH protocol to "2 only"Add rows, columns, scrollback bufferSave sessionsVNC clientRecommended for install of Linux, some softwareRealVNC is described3270 emulatorSet Enter and Clear key if possibleSet to use 43 linesSet to Reconnect after logoffFor Linux, x3270 is most popular© 2012 IBM Corporation14

IBM Systems and Technology GroupIntroduction - Configure a PC serverInstalling Linux on zSeries is a chicken and egg problemRecommendation: install Linux on an Intel box as a temporary NFS server:Install Linux onto a PCCopy files associated with this book to this NFS serverUntar to /var/nfs/SG248147/Set up an install directory:For RHEL: /var/nfs/rhel64/For SLES: /srv/nfs/s11s3/Configure the NFS server to export these two directoriesIf installing z/VM using FTP: copy z/VM to /var/ftp/zvm6x/ or /srv/ftp/zvm6x/© 2012 IBM Corporation15

IBM Systems and Technology GroupInstalling and configuring z/VMObtain z/VM through electronic download (optional)Configure an FTP server for z/VM installation (optional)Install z/VM from DVD or FTP serverCustomize TCPIP - z/VM stack, FTP serverCustomize SYSTEM CONFIGDefine VSWITCHes, other configurationAdd volumes for paging and minidisksCPFORMAT EXEC is includedCreate LNXMAINT for common CMS files- kernels, RAMdisks, PARMfiles, etc.Customize system startup and shutdownSHUTDOWN z/VM signals Linux servers to shutdownIPL of z/VM autologs (boots) important Linux serversz/VM security issues - change default password© 2012 IBM Corporation16

IBM Systems and Technology GroupObtain z/VM through Electronic DownloadGo to the z/VM service page:http://www.vm.ibm.com/service/Click on the link IBM Shopz in the section IBM Support PortalsSign in by clicking on the link Sign in for registered users in the upper rightClick on the link create new software ordersOn Step 1, click on the radio button z/VM Products and choose VM SDO version 6 in thedropdown menu to the right. Click Continue.On Step 2, select a hardware system on which you plan to run z/VMOn Step 3, first filter, select VM - VM Base Product, second filter, select Show allproducts then click Show catalogSelect z/VM V6 3390 System DDR and click ContinueOn Step 4, verify the order and click ContinueOn Step 5, verify the entitlements and click ContinueOn Step 6, for the Preferred media, select Internet and click ContinueOn Step 7, review and click Submit© 2012 IBM Corporation17

IBM Systems and Technology GroupConfigure an FTP server for z/VM installationPrepare the z/VM product install filesInstall the FTP serverConfigure the FTP serverAnonymous or not?Test the anonymous FTP serverAside: interesting fact:gpok240:/nfs # du -sh sles11sp2 rhel6.2 zvm6213G sles11sp25.3G rhel6.24.1G zvm62© 2012 IBM Corporation18

IBM Systems and Technology GroupInstall z/VM from DVD or FTP serverStart the z/VM installImportant screens (below and next chart)Copy a vanilla z/VM system to DASDIPL the first SSI memberNew IPL Parms:==> q iplparmsFN=SYSTEM FT=CONFIG PDNUM=1 PDVOL=D964IPL remaining SSI membersVerify the installationConfigure TCP/IP© 2012 IBM Corporation19

IBM Systems and Technology Groupz/VM install screens (cont'd)© 2012 IBM Corporation20

z/VM 6.2SSI blockdiagramIBM Systems and Technology GroupM01RES M01S01MAINT 123 $SPOOL$ A01M01P01$PAGE$ A01M01W01MAINT 124CEC 1M02RESMAINT 123M02P01$PAGE$ A02M02S01$SPOOL$ A02M02W01MAINT 124LPAR Member 1: 1________SysID:SYS_ID_1 ___________ISFCcommunicationsMember 2SYS_ID_2VMCOM1PMAINT133620RL1MAINT620131620RL2MAINT620132620RL3RELVOL3Member 3SharedDisksMember 4M03RESMAINT 123SYS_ID_3M03S01$SPOOL$ A01CommonLANCEC 2M04RESMAINT 123SYS_ID_4M04S01$SPOOL$ A01M03P01$PAGE$ A01M03W01MAINT 124M04P01$PAGE$ A01M04W01MAINT 124© 2012 IBM Corporation21

IBM Systems and Technology GroupSSI block diagram - values used in book© 2012 IBM Corporation22

IBM Systems and Technology GroupCustomize z/VM TCP/IP stack and FTP serverRecommend IPWIZARD for TCPIP configurationRun once for each SSI memberConfigure XEDIT profile on TCPMAINTRecommend turning on z/VM FTP serverRun once for each SSI memberNew: Attach the z/VM TCP/IP stack to the HA VSWITCHComment out :attach. line in SYSTEM DTCPARMSModify PROFILE TCPIP: OSA rdev => 0600 vdevGrant TCPIP access to VSW1 in user directory© 2012 IBM Corporation23

IBM Systems and Technology GroupCustomize SYSTEM CONFIG fileRecommendationsIncrease retrieve key capacity - from 20 to 99Allow VDISKs to be created for swap spacesUsing SWAPGEN EXEC is common to create in-memory Linux swap spacesTurn off the Disconnect Timeout featureSo Linux virtual machines are not forced off by SYSTEMDefine layer 2 virtual switch (now recommended)Set up "Equivalency IDs" - new for z/VM 6.2/* Add EQID statements for OSA addresses and unique MAC IDs */POKDEV62: beginrdev 4200-420f eqid osaset1 type osardev 4300-430f eqid osaset1 type osavmlan macprefix 02000bPOKDEV62: endPOKTST62: beginrdev 4200-420f eqid osaset1 type osardev 4300-430f eqid osaset1 type osavmlan macprefix 02000cPOKTST62: end© 2012 IBM Corporation24

CPFORMAT EXEC==> cpformatSynopsis:IBM Systems and Technology GroupFormat and label DASD as page, perm, spool or temp disk spaceThe label written to each DASD is W where: is type - P (page), M (perm), S (spool) or T (Temp disk) is the 4 digit addressSyntax is:--CPFORMAT--.-vdev--------.--AS---.-PERM-.---------> att a775-a779 *A775-A779 ATTACHED TO MAINT==> cpformat a775-a779 as page...New: Owner information is added to CP-owned devices© 2012 IBM Corporation25

IBM Systems and Technology GroupAdd volumes for paging and minidisksCopy the CPFORMAT EXECFormat volumes for page spaceUse the CPFORMAT EXEC with "for page"Format DASD for minidisksUse the CPFORMAT EXEC with "for perm"Update the SYSTEM CONFIG file. e.g.:POKDEV62: BEGINCP_Owned Slot 251 JP628ACP_Owned Slot 252 JP6288CP_Owned Slot 253 JP6233CP_Owned Slot 254 JP6232CP_Owned Slot 255 JV6282POKDEV62: ENDPOKTST62: BEGINCP_Owned Slot 251 WP633ECP_Owned Slot 252 WP633CCP_Owned Slot 253 WP633BCP_Owned Slot 254 WP628BCP_Owned Slot 255 WV639DPOKTST62: END...User_Volume_List CV6285 CV6286 CV6287User_Volume_Include JM6*© 2012 IBM Corporation26

IBM Systems and Technology Groupz/VM security issuesChange passwords in USER DIRECTUse a z/VM Security product?IBM RACFCA VM:SecureThe paper z/VM Security and Integrityhttp://www.vm.ibm.com/library/zvmsecint.pdf© 2012 IBM Corporation29

Servicing z/VMIBM Systems and Technology GroupApply a Programming Temporary Fix (PTF)Get service from InternetReceive, apply and buildPut into productionApply a Recommended Service Upgrade (RSU)RSU6202 is now available and documentedDetermining z/VM's service level© 2012 IBM Corporation30

IBM Systems and Technology Groupz/VM Live Guest Relocation (LGR)LGR considerationsUSERs are relocatable, not IDENTITYsMemory size (central, expanded)Link and resource contentionAdd OPTION CHPIDV ONE to the Linux PROFILE in user directoryFCP/SCSI disk devicesLinux must not have CMS disks at relocate timeDisks can be detached at Linux boot timeAdded to /etc/rc.d/rc.local:...chshut halt vmcmd logoffchshut poff vmcmd logoffmodprobe vmcpvmcp det 190vmcp det 191vmcp det 19dvmcp det 19ermmod vmcpRelocate a Linux system==> vmrelocate test ==> vmrelocate move © 2012 IBM Corporation31

IBM Systems and Technology GroupDirMaint, SMAPI and RACFConfigure DirMaintEnable DirMaintTailor DirMaintCustomize the EXTENT CONTROL fileStart DirMaintTest DirMaintConfigure SMAPISet up basic SMAPI configurationTurn off ensemblesStart SMAPI at IPL timeTest SMAPISome common DirMaint tasksRACF with shared database© 2012 IBM Corporation32

IBM Systems and Technology GroupInstall and configure RHEL 6.4 on LNXADMINInstall the golden imageCreate the IDENTITY LNXADMINSet LNXADMIN to start at IPL timePrepare the RHEL 6.4 bootstrap filesInstall RHEL 6.4 LinuxBoot the new system from diskConfigure the Linux administration systemCopy RHEL 6.4 install tree/other files from PC to LNXADMINConfigure yumTurn off unneeded servicesConfigure the VNC serverSet system to halt on SIGNAL SHUTDOWNTurn on NFS serverConfigure SSH keysChange order of swap disksInsert vmcp moduleReboot/verify changes© 2012 IBM Corporation33

IBM Systems and Technology GroupInstall and configure the RHEL 6.4 golden imageInstall the golden imageCreate the RH64GOLD virtual machinePrepare the RH64GOLD parameter filesInstall RHEL 6.4 on the golden imageFile system layout with LVMsVerify the installationConfigure the golden imageConfigure automount of the install treeConfigure yum for online updatesTurn off unneeded servicesConfigure the VNC serverSystem to halt on SIGNAL SHUTDOWNConfigure SSH keys and boot time settingsChange the order of the swap disksReboot system and verify changes© 2012 IBM Corporation34

IBM Systems and Technology GroupConfigure RHEL 6.4 for cloningDefine three new virtual machinesClone a virtual server manuallyClone a virtual server automaticallyReview system status© 2012 IBM Corporation35

Cloning LinuxCloning block diagram:IBM Systems and Technology GroupLinux administration systemGolden imageCloned imageLNXADMIN 100RHEL62 100FLASHCOPYor ddLINUX153 100vmcp LINK r/oclone scriptvmcp LINK r/wRHEL62 101LINUX153 101vmcp LINK r/oPVFLASHCOPYor ddPVVSWITCH VSW1 9.60.18/25© 2012 IBM Corporation36

IBM Systems and Technology GroupCreate RHEL 6.4 appliancesA Web Server applianceAn application development applianceAn LDAP server applianceA file and print server applianceSection on setting up subversion© 2012 IBM Corporation37

IBM Systems and Technology GroupInstall SLES 11 SP3 on LNXADMINReview the identity LNXADMINPrepare the SLES 11 SP3 bootstrap filesInstall SLES 11 SP3 on to LNXADMINConfigure the Linux administration systemReset install locationTurn off unneeded servicesApply serviceInstall the cmsfs packageEnable vmcpSet system to halt on SIGNAL SHUTDOWNModify zipl.confReboot and verify changes© 2012 IBM Corporation38

IBM Systems and Technology GroupInstall the SLES 11 SP3 golden imageCreate the S113GOLD virtual machineCreate the S113GOLD parameter fileInstall the SLES 11 SP3 golden imageLogical volumes for flexibility:Configure SLES 11 SP3 golden imageConfigure the VNC serverPrepare for YaST Online UpdateTurn off unneeded servicesApply service with Online UpdateConfigure /etc/inittabConfigure SSH keysModify zipl.confCleanup temporary filesReboot and verify changes© 2012 IBM Corporation39

IBM Systems and Technology GroupClone SLES 11 SP3Clone a virtual server manuallyClone a virtual server automaticallyUse AutoYASTUse Kiwi© 2012 IBM Corporation40

IBM Systems and Technology GroupCreate SLES 11 SP3 appliancesA Web Server applianceAn LDAP applianceA file and print server applianceAn application development applianceIncluding subversion© 2012 IBM Corporation41

IBM Systems and Technology GroupMonitor and tune z/VM and LinuxUse basic z/VM commandsEspecially: INDICATEThe z/VM Performance ToolkitConfigure the z/VM Performance ToolkitConfigure Web Browser supportConfigure PERFSVMStart the z/VM Performance ToolkitUse the z/VM Performance ToolkitSection on the VIR2REAL EXECCollect and use raw CP Monitor dataMonitor Linux performance data from the kernelMonitor Linux with sysstat© 2012 IBM Corporation42

IBM Systems and Technology GroupNew chapter: Working with disksAdd disk space to virtual machinesAdd a logical volumeExtend an existing logical volumeAdd SCSI/FCP disksAs emuldated devices (aka "EDEVs")As real devicesConfigure multipathingHyperPAV for Linux Use© 2012 IBM Corporation43

IBM Systems and Technology GroupMiscellaneous RecipesRescue a Linux systemSet up memory hot pluggingUtilize the cpuplugd serviceHardware cryptographic support for OpenSSH (remove???)The X window systemCentralizing home directories for LDAP users (no specific steps)Automatically logging root onto the 3270 consoleAdding CTCs to an SSI clusterSetting up Linux Terminal ServerSetting up a private interconnectCreating a hipersocket connection between Linux and z/OSCRON like service virtual machineConfiguring Port Groups with LACP© 2012 IBM Corporation44

ResourcesIBM Systems and Technology GroupAll Virtualization Cookbooks and other papers:http://www.vm.ibm.com/devpages/mikemac/The Linux for zSeries and S/390 portalhttp://linuxvm.org/The linux-390 list serverhttp://www2.marist.edu/htbin/wlvindex?linux-390The IBMVM list serverhttp://www.lsoft.com/scripts/wl.exe?SL1=IBMVM&H=LISTSERV.UARK.EDULinux for zSeries and S/390 developerWorks®http://awlinux1.alphaworks.ibm.com/developerworks/linux390/index.shtmlRed Hat Enterprise Linux evaluationhttp://www.redhat.com/rhel/server/mainframe/SUSE LINUX Enterprise Server evaluationhttp://www.novell.com/products/linuxenterpriseserver/eval.htmlz/VM publicationshttp://www.vm.ibm.com/pubs/z/VM performance tipshttp://www.vm.ibm.com/perf/tips/© 2012 IBM Corporation45

QuestionsAre there any questions?IBM Systems and Technology Group© 2012 IBM Corporation46