UCF Content Transfer Tuning Guide - EMC Community Network

UCF Content Transfer Tuning Guide
This document provides guidance on tuning the performance of the UCF content
transfer mechanism. UCF (Unified Client Facilities) is the EMC Documentum
framework for performing content transfer between the client and content
servers over HTTP.
September 2008

Copyright © 2008 EMC Corporation. All rights reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION
MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE
INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com
All other trademarks used herein are the property of their respective owners.
2

Table of Contents
Introduction .................................................................................................................................. 4
Overview of UCF.......................................................................................................................... 4
High-Level UCF Operations......................................................................................................... 4
Optimizing UCF for Best Performance ........................................................................................ 5
Tip #1 – Use ACS and BOCS for Content Transfer Whenever Possible................................. 5
Tip #2 – Optimize UCF Client Initialization............................................................................... 5
Tip #3 – Configure On-Demand Virus Scanners to exclude unnecessary files ....................... 6
Tip #4 – Reduce UCF Client Communications over High Latency .......................................... 6
Tip #5 – Tune the Buffer or Chunk Size to Improve Download over High Latency ................. 7
Tip #6 – Improve Upload Performance in D6 and Above ........................................................ 9
Tip #7 – Use Adaptive Parallel Content Transfer to Consume More Available Bandwidth ..... 9
Tip #8 - Tuning Content Download when Documentum User Directory is on a Network Drive
................................................................................................................................................ 11
Tip #9 – Use UCF Client Logging to Measure performance .................................................. 11
Appendix A: Reduce the Effect of On-Access Scanning on UCF Initialization and Other Java
Applets After Reboot.................................................................................................................. 12
Appendix B – Changing UCF Client Settings for All Users........................................................ 15
3

Introduction
In Documentum applications, the speed at which users can retrieve or add content to the system plays a big
part in the overall end user experience. Network latency and bandwidth restrictions, physical architecture
and client configuration all have an impact on the time it takes between initiating a content transfer,
performing the upload or download, and returning control back to the user.
This document will discuss various tuning options available to achieve the best possible performance for
UCF content transfer in your network environment.
Overview of UCF
UCF (Unified Client Facilities) is the EMC Documentum framework for performing content transfer
between the client and content servers over HTTP. It provides many features that extend beyond simple file
transfer protocol, such as:
- Content compression to optimize transfers of larger files over the network
- Support for complex documents, such as XML files
- Support for compound document types, such as OLE objects inside a Microsoft Office document
- Awareness of locally downloaded files, and the ability to avoid downloading the same content
twice
- Registry support, to allow checkout from one application and check in from another
- Recoverability in the case of a brief network interruption
- Location aware, to allow users to transfer content from the closest ACS or BOCS server
High-Level UCF Operations
When a content transfer is initiated, the UCF client running on the end user machine is initiated if it is not
already running. The applet loads the UCF client jars and initiates contact with the UCF server, running on
the application server.
The UCF client and server pass information back and forth about the environment and requested action
before content is transferred between the two machines. Depending on the architecture and configuration,
content may be transferred through the application server, directly to or from an ACS server on the content
server, or a BOCS server located near the user.
After the content transfer is complete, there will be some additional communication from server to client to
provide instructions on registry entries to be added or modified, directions on how to launch the appropriate
application for view or edit operations, or instructions on how to delete the local file on check in.
4

Optimizing UCF for Best Performance
Tip #1 – Use ACS and BOCS for Content Transfer Whenever Possible
The introduction of ACS and BOCS servers in 5.3 SP1 have allowed content transfer to be performed
directly from the content server, rather than requiring that the content always pass through the application
server. This not only removes the double-hop that is required (which is very costly for small files) but also
reduces the load on the application server.
In the 5.3 timeframe, ACS and BOCS could only be used for download operations such as checkout, export
and view. All upload operations still passed through the application server.
From D6 onward, ACS and BOCS servers can also be used for upload operations.
Note that not all applications are network location aware and able to take advantage of remote ACS and
BOCS servers.
Tip #2 – Optimize UCF Client Initialization
In 5.3 SP1 through SP5, D6 and D6SP1, the UCF client engine will time out after one minute of inactivity,
and the javaw.exe process would be terminated. This means that if a user initiates another content transfer
request after the timeout has occurred, the browser must re-initialize the UCF client and restart javaw.exe.
This value can be increased in the ucf.client.config.xml file as follows:
Edit the ucf.client.config.xml file (located at C:\Documents and
Settings\\Documentum\ucf\\shared\config\ucf.client.config.xml) and add the
following option to extend the UCF Client Timeout.
30
Close all browser windows and log back into the application. The next content transfer operation will cause
these new settings to take effect.
With these new settings, the UCF client will remain alive for at least 30 minutes of inactivity before
terminating itself.
Note that the default value has been increased in 5.3 SP6 and D6.5.
5

Tip #3 – Configure On-Demand Virus Scanners to exclude unnecessary files
The first time the UCF client is initialized after restart, the on-demand scanner will need to scan all jars and
DLLs prior to launching them. In addition, if it is the first time java has been used since reboot, the scanner
will scan all of the Java jars and DLLs. This slows initialization down tremendously.
With some small changes to the configuration of the virus scanner, it is possible to see much faster UCF
client initialization without sacrificing the security of the system. Please see Appendix A for more detailed
instructions on how to implement this recommendation when using McAfee.
Tip #4 – Reduce UCF Client Communications over High Latency
When the UCF client is remote from the UCF server, the time it takes for each request/response is greatly
increased. As there can be many such roundtrips, it can add significant time to the end-to-end operation,
which is especially noticeable for small files.
In order to reduce the chattiness of the UCF communication, handshake and request optimizations were
introduced in D6 SP1 and 5.3 SP6.
Note: These optimizations were also back ported to 5.3 SP5 and can be found at
ftp://dev_pre:qa5.grN6@ftp2.documentum.com/UCF/5.3SP5/SP5_HF_GW2. However, EMC
strongly recommends that customers upgrade to 5.3 SP6 instead of applying the hotfix as 5.3 SP6 includes
a number of other performance and stability enhancements.
6

Tip #5 – Tune the Buffer or Chunk Size to Improve Download over High Latency
Each application server has a default sized buffer that it uses when transferring data across the network to
clients. In most cases, this buffer is sized much too small, resulting in slower transfers over the WAN as
more roundtrips are necessary.
Tomcat and Jboss
If you are using Tomcat or Jboss as the application server and you are not using ACS, you can add the
socketBuffer parameter to the HTTP Connector as shown below:
If you are using 5.3 ACS and BOCS the application server is Tomcat 5.0.28. In versions 5.3 SP1 or SP2,
you will need to add the socketBuffer parameter. In 5.3 SP3 through SP6, this parameter should already
have been added by the installer.
In 6.5 and beyond, Jboss is the container for ACS and BOCS. In 6.5 SP1, enhancements were added to the
embedded Jboss server to enable it to recognize and use the socketBuffer parameter.
7

BEA Weblogic
If you are using BEA Weblogic as your application server and you are not using ACS, be sure to add the –
Dweblogic.ChunkSize parameter. By default, the ChunkSize is 4k which is much too small.
It can be set in the config.xml for the domain:
server1
false
localhost
17001
-Dweblogic.Chunksize=64000
weblogic
{3DES}wmzrpHOymvTy3q8TOeF1qQ==
It can also be defined in the set_domain_env batch file with the other JAVA_PROPERTIES values:
set JAVA_PROPERTIES=-Dplatform.home=%WL_HOME% \
-Dwls.home=%WLS_HOME% -Dwli.home=%WLI_HOME% \
-Dweblogic.Chunksize=64000
In D6 and D6 SP1, the application server that hosts ACS and BOCS is BEA Weblogic, and the installer has
automatically added those options to the script that starts the embedded Weblogic server
(startMethodServer.cmd for example).
IBM Websphere
If you are not using ACS and content is being transferred through WAS, then you may see improvements in
content download performance for large files by modifying the ResponseChunkSize configuration
parameter in the plugin-cfg.xml.
where N equal the chunk size in kilobytes. The default value is 64 (kilobytes).
8

Tip #6 – Improve Upload Performance in D6 and Above
In D6, a parameter was added to the ucf.client.config.xml file that allowed the users to specify a chunk size
to be used when uploading content.
Internal testing showed that a value of 49152 worked best with ACS on BEA when simulating various
WAN conditions. However, in environments where customers are not using ACS write, or whose network
conditions are different from that which was tested, this value may not be optimal. Increasing or decreasing
the optimal.chunk.size parameter based on your specific network conditions can be beneficial.
49152
Note that this setting is ignored if the client is using Sun Java version 1.6 due to a Java bug
(http://bugs.sun.com/bugdatabase/view_bug.dobug_id=6631048).
Tip #7 – Use Adaptive Parallel Content Transfer to Consume More Available
Bandwidth
By default in most versions of UCF, content download operations will be done in a single thread. If
sufficient network bandwidth is available, the download operations could perform significantly faster if the
file was broken up into smaller pieces and transferred by multiple concurrent threads.
In 5.3 SP6 and D6.5 the UCF client can be configured to use multiple threads when downloading files. This
is controlled in the ucf.client.config.xml file using the following parameters:
5
131072
300
131072
The “max.parallel.download.streams” parameter limits the number of threads that can be run concurrently
when performing a parallel download. In this example, 5 separate streams could potentially be initialized. If
one of the streams finishes their assigned task ahead of the others, it is then free to request a new range of
bytes to be downloaded.
The “min.parallel.segment.size” parameter specifies that if the remaining portion of a document after a byte
range is assigned is smaller than a specified value, it should not be assigned to a new thread. Rather the
original thread’s byte range should be extended to include those additional bytes. For example, if a thread is
supposed to download 500Kb of a 600Kb file, as the remaining number of bytes is less than 128k, no new
thread will be started and the original thread will assume ownership of those additional bytes.
The “Adaptive” part of parallel content transfer is controlled by the remaining two parameters. As tests
have shown that disk I/O actually becomes a bottleneck and degrades performance if the content is
transferred in parallel over LAN conditions, it is important to be able to control when the parallel content
transfer is actually turned on. In this case, the UCF client will measure how many bytes were transferred
9

initially by a single thread within a specified timeframe. If the number of bytes is less than what it
expected, parallel content transfer will be turned on. If it is more than was expected, then it is assumed that
bandwidth and latency are sufficient for a single thread to transfer the content most efficiently.
In the example above, the UCF client will measure the number of bytes that have been downloaded in the
first 300ms of transfer (or as close to that time as possible). It will enable parallel content transfer if less
than 131072 bytes (128k) have been downloaded, and disable it if more than 128k has been downloaded.
If your remote users have very high latency, it is likely that you will want to increase the
“measurement.time.interval” to match the round trip time plus time to download a small portion of the file.
For example, if you have 200ms round-trip latency, you might consider increasing the
measurement.time.interval to 500ms to compensate for the time it takes for the request to reach the ACS
server and start the content transfer.
In order to permanently disable parallel content transfer, all that must be done is changing the
“max.parallel.download.streams” to 1, or decreasing the “single.thread.throughput” to a very small number,
such as 1. In this case, regardless of the actual throughput, parallel content transfer will not be used. This is
especially important in load testing scenarios, where many clients are being simulated from the same client
machine.
10

Tip #8 - Tuning Content Download when Documentum User Directory is on a
Network Drive
If the location of the Viewed and Checkout directories are on a network drive, or users regularly export
content to a network drive, performance can be improved by setting the value of “ucf.file.buffer.size” to a
larger number. The default value is 32768.
To set it for a single user, simply add the following to the ucf.client.config.xml on the client machine:
61440
To set the default for all users, add the same option to the ucf.installer.config.xml file on the application
server and force a new version of UCF on the server side using the steps in Appendix B.
Tip #9 – Use UCF Client Logging to Measure performance
UCF Client logs are extremely useful when diagnosing UCF performance.
UCF client logging is enabled on the end user’s machine.
Edit the ucf.client.logging.properties file, located at:
C:\Documents and Settings\\Documentum\ucf\\shared\config
For best results, set the log level to FINEST, as shown below:
handlers=java.util.logging.FileHandler, java.util.logging.ConsoleHandler
.level=FINEST
#-------------------
java.util.logging.FileHandler.pattern=C:/Documentum/logs/ucf.client%u.log
java.util.logging.FileHandler.limit=10485760
java.util.logging.FileHandler.count=10
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.FileHandler.encoding=UTF-8
#-------------------
java.util.logging.ConsoleHandler.level=OFF
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
In addition, to prevent the UCF client log from overwriting itself each time, change the value of
java.util.logging.FileHandler.count to a number greater than 1. As each new client initializes, the old logs
will be renamed and preserved for further analysis.
In the UCF client logs you will see entries such as “Logging request: getFile” and “Handled request:
getFile” with timestamps. This can be used to better measure the time taken in the UCF pre-processing,
content transfer and post-processing phases.
11

Appendix A: Reduce the Effect of On-Access Scanning
on UCF Initialization and Other Java Applets After Reboot
1. Right-click on the VirusScan icon in the system tray and choose “On Access Scan Properties.
2. Click on All Processes.
3. Click on Advanced.
4. Ensure “Scan inside archives” is NOT selected.
5. Click on the Detection tab.
6. Click on the Exclusions button.
12

7. Click on Add.
8. Add C:\Documents and Settings\s38737\Documentum\ucf , with “Also exclude subfolders”, “On
Read” and “On Write”.
9. Click on OK to save.
10. Click Add to add another path.
13

11. Add C:\Program Files\Java with “Also exclude subfolders” and “On Read”.
12. Click OK to save.
14

Appendix B – Changing UCF Client Settings for All Users
1. Edit the \webtop\wdk\contentXfer\ucf.installer.config.xml file.
2. In this file, you will find the following line,
3. Change the highlighted text above from 5.3.0.XXX to 5.3.0.XXXa
4. Under the configurations tag, change or add the desired option.
value
The next time the UCF client engine is initialized on the client side, the new UCF settings will be
downloaded and added to the ucf.client.config.xml file.
There may be cases where the “default” values in the ucf.installer.config.xml should not be applied to the
client side. In this case, the “persistent” attribute can be added to the option on the client side. This will
prevent any changes on the server side from being applied to that particular client.
value
If not specified, then persistent is set to false.
The persistent property can be specified on the ucf.installer.config.xml and in the ucf.client.config.xml. The
following chart outlines the expected behavior:
Server Client Description
false false Server value will override client value.
true true Server value will override client value.
true true Server value will override client value.
false true Client value will not be overridden.
15