JADE Documentation (PDF) - SOS-Berlin

Software- und Organisations-Service GmbH 

Managed File Transfer 

SOSFTP 

Documentation 

Technical Manual 

23 July 2010 

Software- und Organisations-Service GmbH 23 July 2010

SOSFTP Documentation 2 

1 Yet another File Transfer Client? ..........................................................................................................3 

2 Installation ............................................................................................................................................4 

2.1 Overview........................................................................................................................................... 4 

2.2 Software Requirements..................................................................................................................... 5 

2.3 SOSFTP Client ................................................................................................................................. 5 

2.4 SOSFTP Server ................................................................................................................................ 6 

2.5 Web GUI........................................................................................................................................... 7 

2.6 SOSFTP Reporting ........................................................................................................................... 9 

3 Architecture ........................................................................................................................................10 

3.1 SOSFTP Client ............................................................................................................................... 10 

3.1.1 Simple Transfer........................................................................................................................... 10 

3.1.2 Jump Transfer............................................................................................................................. 11 

3.1.3 Log Output.................................................................................................................................. 12 

3.2 SOSFTP Server and Web GUI ........................................................................................................ 13 

3.2.1 Importing the Transfer History ..................................................................................................... 14 

3.2.2 Quick View with the Web GUI ..................................................................................................... 16 

3.3 SOSFTP Reporting ......................................................................................................................... 17 

4 Use Cases Client................................................................................................................................18 

4.1 Configuration with Profiles............................................................................................................... 18 

4.2 Simple Transfer............................................................................................................................... 19 

4.3 Install Operation.............................................................................................................................. 23 

4.4 Jump Transfer................................................................................................................................. 24 

4.5 Polling for files................................................................................................................................. 27 

4.6 Logging........................................................................................................................................... 28 

4.6.1 Transfer Log ............................................................................................................................... 28 

4.6.2 Transfer History .......................................................................................................................... 28 

4.7 Dynamic Password Retrieval........................................................................................................... 30 

5 Use Cases Server ..............................................................................................................................31 

5.1 History Import.................................................................................................................................. 31 

5.1.1 Job Configuration........................................................................................................................ 32 

5.1.2 Setting up orders for receiving history files .................................................................................. 35 

5.2 Custom Fields ................................................................................................................................. 36 

5.2.1 SOSFTP Client ........................................................................................................................... 36 

5.2.2 Web GUI..................................................................................................................................... 37 

5.3 SOSFTP Reporting ......................................................................................................................... 39 

5.3.1 Configuration .............................................................................................................................. 39 

5.3.2 Generating a Report.................................................................................................................... 41 

6 Complete Client Parameter Reference................................................................................................44 

Software- und Organisations-Service GmbH


1 Yet another File Transfer Client? 

SOSFTP is a client/server solution for managed file transfer. SOSFTP addresses the gaps of individual solutions 

that are identified in the fields of implementation, logging and reporting: 

� Concretely, the problems with shell script implementations are missing error detection and error handling. 

� Additionally, individual shell script implementations result in individually organized configurations that are 

hard to maintain. 

� Protocols generated by shell scripts are individually arranged. This leads to log files that are hardly understandable 

at a first glance. Error states often are not clearly perceivable in such reports. 

� There is no central information about the files sent and received per destination and there is no individual 

reporting for each destination. There is no overview about the rate of errors produced by file transfers based 

on specific shell scripts or destinations. 

Therefore, SOSFTP implements the following features: 

� There is a standard implementation in Java (JRE 1.6 or higher required) for all platforms without any further 

installation prerequisites. SOSFTP has been tested for Windows 2000/2003/2008/XP/Vista, Solaris, AIX, 

HP-UX, Linux and should be operational for additional platforms that are capable of running a JRE. 

� The SOSFTP client supports the protocols FTP, FTPS 1 (FTP over SSL) and SFTP. 

� For all protocols username/password authentication is available and for the SFTP protocol public/private key 

authentication is supported. 

� In addition to simple file transfer from a client to a server and vice versa within one TCP/IP network it is possible 

to transfer files from one host in the intranet through a server in the Demilitarized Zone called “Jump 

Host” to another host in the internet and vice versa. Thus it is possible to create a bridge between two independent 

TCP/IP networks for direct file transfer. 

� Detailed error detection and error handling are supported. The configuration can be organized by command 

line and by a configuration file. 

� For logging purposes standardized protocols in a structured and configurable format, a history of all transfers 

and automated import of the log information into a database are supported. Therefore, network monitors for 

automated alerting in case of errors can be easily connected. 

� SOSFTP provides a Web GUI for constant checking and for the history of file transfers. 

� Automated generation of reports, e.g. configurable KPI reports, and their delivery by email are supported. 

Advantages 

The advantage of the SOSFTP logging and reporting capabilities is an increase in efficiency: 

� All relevant information about the file transfer history can be stored automatically in a central database. 

� Via a graphical web interface (Web GUI) you can see at a first glance the important information on successful 

and failed transfers. 

� Application Managers do not have to actively survey all protocols. There is no time-consuming searching for 

logs that are stored on different remote hosts. No direct access is required to each of those server systems 

for Application Managers in order to check the transfer status. No manual log analysis is required. 

� Automatically generated log reports provide the relevant information in a compact form. 

1 Only implicit FTPS (using a dedicated FTPS port) is supported. Explicit FTPS (Server uses the same port for 

FTP and FTPS) is not supported. 



� SOSFTP can automatically detect errors and will then alert the Application Manager by sending an alert 

email. 

2 Installation 

2.1 Overview 

The SOSFTP Software contains the following packages: 

� SOSFTP Client 

A Java based FTP/SFTP-Client for Unix and Windows for managed file transfer supporting secured file 

transfer between different TCP/IP networks. The SOSFTP Client supports the protocols FTP, FTPS 2 

(FTP over SSL) and SFTP. This package can be used as a standalone application, it is not dependent 

on the other packages. 

� SOSFTP Server 

Add-on for the Job Scheduler. The SOSFTP Server collects the information of file transfers processed 

by any number of SOSFTP Clients. The SOSFTP Server manages the information in a database (Transfer 

History Database). 

� Web GUI 

This package supports a quick survey of managed file transfers stored in the Transfer History Database. 

The GUI is running as web application in a web browser. This package requires the SOSFTP Server. 

� SOSFTP Reporting 

This package supports the automatic generation of reports. The reports contain the summary of information 

about managed file transfers. The reports can be automatically send by email. This package requires 

the SOSFTP Server. 

2 Only implicit FTPS (using a dedicated FTPS port) is supported. Explicit FTPS (Server uses the same port for 

FTP and FTPS) is not supported. 



2.2 Software Requirements 

� The SOSFTP Client requires JRE 1.6 or higher. 

� The SOSFTP Server requires JRE 1.6 or higher, a Job Scheduler (http://jobscheduler.sourceforge.net) 

and a database to store the file transfer information. 

The supported database systems are 

- DB2 8.x,9.x 

- Oracle 8.1.7, 9.2, 10g. 

- SQL Server 2000, 2005, 2008 

- Sybase ASE 15 

- MySQL 4.1, 5.x 

- PostgreSQL 8.x 

- Firebird 2.0 

� The Web GUI requires a web server e.g. Apache and PHP 4 or PHP 5. 

2.3 SOSFTP Client 

The package SOSFTP Client is delivered in the sosftp_client/ directory with the following files and subdirectories: 

� doc/ (Directory of the documentation) 

o sosftp.xml (Documentation of the SOSFTP Client) 

o sosftp.xsl (Stylesheet for sosftp.xml) 

o banner_english.gif (Image for sosftp.xml) 

o banner_german.gif (Image for sosftp.xml) 

o sosftp.cmd (Startscript for Windows) 

o sosftp.sh (Startscript for Unix) 

o com.sos.net-1.6-[date]-[svnr].jar (Library containing the relevant Java classes) 

o com.sos.settings-1.6-[date]-[svnr].jar (Library containing the relevant Java classes) 

o com.sos.util-1.6-[date]-[svnr].jar (Library containing the relevant Java classes) 

o com.sos.connection-1.6-[date]-[svnr].jar (Library containing the relevant Java classes) 

o com.sos.configuration-1.6-[date]-[svnr].jar(Library containing the relevant Java classes) 

o com.sos.xml-1.6-[date]-[svnr].jar (Library containing the relevant Java classes) 

o xalan.jar (Third party software component) 

o commons-net-1.2.2.jar (Third party software component) 

o trilead-ssh2-build211.jar (Third party software component) 

o readme.txt (Configuration instructions) 

o ThirdParty.txt (Licensing information for third party software components) 



To install the SOSFTP Client just copy the sosftp_client/ directory containing files to any directory. In Addition 

if you already have a copy of the SOSFTP Client you can use the operation install of the SOSFTP Client. 

This operation creates another default installation of a SOSFTP Client. For further information see the chapter 

Install Operation. 

2.4 SOSFTP Server 

A regular installation of a Job Scheduler is required. For further information about the installation and configuration 

of the Job Scheduler see the documentation about Installation and Configuration at 

http://www.sos-berlin.com/doc/en/scheduler_installation.pdf. The SOSFTP Server consists of the files mentioned 

below. These files are delivered in the directory sosftp_server/scheduler. 

To install the SOSFTP Server copy the delivered files into the installation directory of your Job Scheduler. 

For the files their appropriate paths in the Job Scheduler installation are given where [install_path] is the 

installation path of the Job Scheduler.: 

� [install_path]/config/live Configuration directory 

o scheduler_sosftp_history.job_chain.xml 

Configuration file relevant for the Import 

o scheduler_sosftp_history.params.xml 

Database parameters for the Import Job 

o scheduler_sosftp_history_file_order.job_chain.xml 

Configuration file relevant for the import 

o scheduler_sosftp_history_receive,receive.order.xml_sample 

Configuration file relevant for the Receive from SOSFTP Client History files 

After the conformation rename this file to 

scheduler_sosftp_history_receive,receive.order.xml 

o scheduler_sosftp_history_receive.job.xml 

Receive-Job Configuration 

o scheduler_sosftp_history_receive.job_chain.xml 

Configuration file relevant for the Receive 

o scheduler_sosftp_import.job.xml 


Import-Job Configuration 

� [install_path]/db/ (Create table scripts) 

o /sosftphistory.sql (Create table scripts) 

� [install_path]/files/ (Default directory for transfer history files) 

� [install_path]/jobs/ (Job documentations) 

o SOSFTPHistoryJob.xml (Job documentation of the Import Job) 

� [install_path]/lib/ (Directory for the libraries) 

o com.sos.ftphistory-1.6-[date]-[svnr].jar (Contains the relevant Java classes) 

The SOSFTP Server requires a database for importing the transfer history. This does not necessarily have to be 

the same database that is used by the Job Scheduler. To install the Transfer History Database execute the appropriate 

create table script in your SQL Client. Otherwise if you skip the installation of the Transfer History Database 

the Import Job will automatically create the required tables in the Job Scheduler database.


Configuration 

1. Configure the SOSFTP Servers database. 

The configuration file scheduler_sosftp_history.params.xml specifies the database parameters that are 

used by the Import Job. If the given database does not exist or if this configuration file does not exist then the 

Import Job will automatically use the Job Scheduler database which is configured in the setting [install_path]/config/sos_settings.ini. 

The SOSFTP Server creates the following tables in the Job 

Scheduler database: 

� SOSFTP_FILES 

� SOSFTP_FILES_HISTORY 

� SOSFTP_FILES_POSITIONS 

2.5 Web GUI 

The Web GUI needs access to the Transfer History Database therefore it is dependent on an installation of the 

SOSFTP Server. Furthermore a web server installation (e.g. Apache) with PHP 4 or PHP 5 is required. The Web 

GUI is delivered in the directory sosftp_web/ with the following files and subdirectories: 

� custom/ (Contains the configuration file custom.inc.php) 

� db/ (Contains database specific files) 

� js/ (Contains JavaScript files) 

� languages/ (Contains language specific files) 

� packages/ (Contains program files) 

� quicklist.css (Used cascading style sheet) 

� site.css (Used cascading style sheet) 

� sosftp_history.php (The main PHP script) 



Install the Web GUI by copying the directory sosftp_web/ with the files shown above in the htdocs/ directory 

of your Apache web server. To configure the Web GUI open the configuration file 

sosftp_web/custom/custom.inc.php in a text editor. 

1. Choose the connection class for your database 

2. Choose the database authorization parameters for your database 

… 

// connection classes for database support (Oracle, SQL Server, MySQL, ...) 

# if (!defined('APP_CONNECTION_CLASS')) 

#{ define ( 'APP_CONNECTION_CLASS', 'sos_oracle_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_mssql_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_mysql_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_odbc_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_pgsql_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_db2_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_fbsql_record_connection' ); } 


#{ define ( 'APP_CONNECTION_CLASS', 'sos_sybase_record_connection' ); } 

if (!defined('APP_CONNECTION_CLASS')) 

{ define ( 'APP_CONNECTION_CLASS', 'sos_oracle_record_connection' ); } 

// database authorization 

if (!defined('APP_CONNECTION_AUTH')) 

{ define ( 'APP_CONNECTION_AUTH', 

'-db=test -user=test -password=test –host=localhost:1521' ); 

} 

?> 

Start the Web GUI by entering “http://[host:port]/sosftp_web/sosftp_history.php” into the address 

bar of your web browser where [host:port] is the host name and port number of your web server installation. 



2.6 SOSFTP Reporting 

A regular installation of a Job Scheduler is required. For further information about the installation and configuration 

of the Job Scheduler see the documentation about Installation and Configuration at 

http://www.sos-berlin.com/doc/en/scheduler_installation.pdf. SOSFTP Reporting needs access to the Transfer 

History Database therefore this package is dependent on an installation of the SOSFTP Server. SOSFTP Reporting 

consists of the files mentioned below. These files are delivered in the directory 

sosftp_reports/scheduler/. To install SOSFTP Reporting copy the delivered files into the installation 

directory of your Job Scheduler. For the files their appropriate paths in the Job Scheduler installation are given 

where [install_path] is the installation path of the Job Scheduler.: 

� [install_path]/config/live (Directory for configuration files) 

o jasper_report_sosftp.job.xml (Configuration file of the Reporting Jobs) 

o jasper_report_sosftp_repeat_for_mandator.job.xml 

� [install_path]/jasperreports/ (JasperReports files) 

� [install_path]/jasperreports/config (Configuration for JasperReports) 


o sos_settings_sosftp.ini (Database connection settings) 

o sosftp_mandator_monthly.jasper (Compilation of the jrxml file) 

o sosftp_mandator_monthly.jrxml (JasperReports definition) 

o sosftp_monthly.jasper (Compilation of the jrxml file) 

o sosftp_monthly.jrxml (JasperReports definition) 

o sosftp_oracle_mandator_monthly.sql (SQL-script for the mandator reports) 

o sosftp_oracle_mandator_monthly_names.sql (SQL-script for mandator names) 

o sosftp_oracle_monthly.sql (SQL-script for the complete report) 

� [install_path]/jasperreports/reports (Empty directory for the reports) 

� [install_path]/jobs/ (Directory for Job documentations) 

o JobSchedulerJasperReportJob.xml (Application Manager Job doc.) 

o JobSchedulerJasperReportJobRepeat.xml (Mandator Job documentation) 

� [install_path]/lib/ (Directory for the libraries) 

o bcmail-jdk13-115.jar 

o commons-collections-2.1.jar 

o commons-digester-1.7.jar 

o commons-javaflow-20060411.jar 

o itext-1.3.1.jar 

o jasperreports-1.2.5.jar 

o poi-2.0-final-20040126.jar 

o sos.documentfactory.jar 

o sos.stacks.jar


3 Architecture 

3.1 SOSFTP Client 

3.1.1 Simple Transfer 

Figure 1: Simple file transfer 

A simple transfer is a direct data transfer within one TCP/IP network. The SOSFTP Client is configured and executed 

on one host while an FTP or SFTP server in the same network is required. 

The SOSFTP Client supports the following operations: 

� The functionality of the operations send and receive is given in Figure 1. 

� The operation send allows the upload of files from one host to the FTP/SFTP server. 

� The operation receive is used for the download of files from the server to the host. 

� The operation remove allows the removal of files on the server. 

� The operation execute is available when using a SSH server. It allows the execution of a command by 

SSH on the remote host. 

� The operation install installs a copy of the SOSFTP Client on a remote host. 

The prerequisite for the Jump Transfer is an installation of the SOSFTP Client on a remote host in the DMZ. 

Such a remote host is called a Jump Host. 



3.1.2 Jump Transfer 

The following schema illustrates the use of jump transfers: 

Figure 2: Sending files by Jump Transfer 

Compared to the Simple Transfer the Jump Transfer is a more advanced mechanism that allows files to be 

transferred between two different TCP/IP networks, i.e. intranet and internet which are connected by a Demilitarized 

Zone (DMZ). The DMZ contains one ore more servers which are protected by package filters to the connected 

networks. 

Due to the level of protection usually there is no direct file transfer enabled by default between the intranet and 

the internet through the DMZ. With the SOSFTP Client this is easy to achieve without creating security holes. 

The SOSFTP Client can act as a bridge. For file transfers through the DMZ the only requirement is to install a 

copy of the SOSFTP Client on a host in the DMZ. Furthermore such a host is called a Jump Host. An installation 

on a Jump Host is effected by using the install operation of SOSFTP. 

� For sending files by Jump Transfer the SOSFTP Client on the local host sends the files to the Jump Host 

and configures the Jump Host to send the files to the FTP/SFTP server on the remote host in the internet. 

� For receiving files by Jump Transfer the SOSFTP Client on the local host configures the Jump Host to 

receive files from the FTP/SFTP server on the remote host in the internet and to send the files back to 

the local host in the intranet. 

The Jump Host is exclusively configured by the SOSFTP client at the local host. No sensitive data like passwords 

are stored on the Jump Host except for private key files that are possibly used to access the remote host. 



Figure 3: Receiving files by Jump Transfer 

3.1.3 Log Output 

By default the SOSFTP Client logs any output to stdout. This output can be redirected into a log file. Additionally 

a transfer history file in csv format can be created. The transfer of each individual file results in a new entry to the 

transfer history file. 

The SOSFTP Server can be configured to pull the transfer history file and to import the history into a database 

(Transfer History Database). In addition, the information on individual file transfers is sent by UDP to the 

SOSFTP Server. 



3.2 SOSFTP Server and Web GUI 

Figure 4: Interaction between SOSFTP Server, SOSFTP Client and Web GUI 

What is the SOSFTP Server? 

The SOSFTP Server is a specific configuration of a Job Scheduler instance. 

This Job Scheduler installation offers multiple automated functions for managing the transfer history of multiple 

SOSFTP Clients. This functionality includes: 

� Import the file transfer history information of the SOSFTP Clients into a central database 

� Analyse the collected transfer information 

� Create reports on the transfer history by using an open source stack with JasperReports 



Figure 5: Importing the Transfer History Data by two alternative ways 

3.2.1 Importing the Transfer History 

The primary task of the SOSFTP Server is to collect the transfer history created by the SOSFTP Clients and to 

store this information into the Transfer History Database. This step is required for subsequent data analysis. 

There are two ways how to import the file transfer history into the database: 

Figure 6: Jobs and Job Chains for importing the File Transfer History 



Import by Receiving Remote Files: 

Periodically the Job Scheduler creates an order for each defined SOSFTP Client to start the Job Chain 1 

(scheduler_sosftp_history_receive) which contains the Collect Job 

(scheduler_sosftp_history_receive). For each order the Collect Job receives by FTP/SFTP the transfer 

history files from the respective SOSFTP Client. The transfer history files are then stored in a local directory 

which is monitored by Job Chain 2 (scheduler_sosftp_history_file_order). Job Chain 2 implements 

the Import Job (scheduler_sosftp_import). For each transfer history file in the monitored directory a file 

order is automatically created by the Job Scheduler and the Import Job imports the information given in the respective 

transfer history files into the database. With the import being completed the local file copies are removed 

from the monitored directory. 

Import by Receiving UDP Messages: 

A SOSFTP Client is configured to actively send an UDP message which contains the information for each file 

that is being transferred. If one file transfer is executed for multiple files, then for each file a separate UDP message 

is being sent The UDP message is an order for Job Chain 3 (scheduler_sosftp_history) which contains 

the Import Job (scheduler_sosftp_import). 



3.2.2 Quick View with the Web GUI 

The Web GUI is an user interface that allows you to monitor the file transfer information that is stored in the database. 

To the Application Manager it offers the possibility to monitor the transfer information for each mandator 

separately. In detail the transfer information can be filtered by mandator, file name, error state (success/error), 

transfer operation (send, receive), transfer protocol, source and target host and by time interval. 

Figure 7: A browser window showing the Web GUI 

� If the checkbox “only last state” is checked then the listing is constrained to show only the last transaction 

of a file. A file is defined as the combination of source path and file name, i.e. the sources “sample1/a.xml” 

and “sample2/a.xml” are two different files. 

� If the checkbox “only last state” is not checked for a file then all transfers in the given time interval are 

listed. 

� If the checkbox “Refresh interval (all 60 s)“ is checked then the view is refreshed automatically every 

minute. You can refresh the view manually by hitting the “search” button. 




Figure 8: Pipeline for Automatic Reporting 

Once the file transfer informations has been collected in a database one way for monitoring is by using the Web 

GUI as described above. Another way for monitoring are automatically generated reports. Such a report can be 

stored as one or more files. The supported file formats are PDF, HTML, RTF, XML and XLS. Additionally either 

the complete Report or just a simple notification can automatically be sent by email to the application manager. 

The Report is created by using JasperReports jobs that are executed by the Job Scheduler of the SOSFTP 

Server. There are two jobs supporting the generation of reports: 

1. JobSchedulerJasperReportJob 

This job receives the information for all file transfers from the database and generates a report. This report can 

be stored in multiple file formats simultaneously. Optional the application manager is informed by email. 

2. JobSchedulerJasperReportJobRepeat 

This job receives the names of all different mandators from the database. For each mandator name the job Job- 

SchedulerJasperReportJob is parameterized individually to generate an individual report. For each mandator 

separate report files are stored with the name of the mandator as its prefix. Again an individual report can be 

stored in multiple file formats simultaneously. Optional the application manager is informed by email. 



4 Use Cases Client 

The SOSFTP Client is executed by calling the SOSFTP command line script. For UNIX this is sosftp.sh and 

for Windows this is sosftp.cmd. The script is configured by command line arguments. No specific order is required 

for the arguments. 

Example 1 

sosftp.cmd -protocol=ftp -operation=send -host=8of9 -user=sos -password=sos 

-local_dir=samples\outbound\ a1.xml b1.xml c1.xml 

-remote_dir=\remote\sosftp\inbound\ 

Sends the three files a1.xml, b1.xml and c1.xml from the current local subdirectory samples\outbound\ 

to the remote directory \remote\inbound\ on the FTP-Server by using the FTP protocol. 

Example 2 

sosftp.cmd -protocol=ftp -operation=send -host=8of9 -user=sos -password=sos 

-local_dir=samples\outbound\ -file_spec=.* 

-remote_dir=\remote\sosftp\inbound\ 

Sends all files matching the regular expression .* from the current local subdirectory samples\outbound\ to 

the remote directory \remote\inbound\ on the FTP-Server by using the FTP protocol. 

4.1 Configuration with Profiles 

For better handling of the configuration all parts of the parameterization except for the parameter –operation 

can by outsourced in a configuration file. This is a text only file which has the following structure: 

;comment 

[profile name] 

;comment 

key=value 

A profile section can contain any valid arguments of the command line script. The respective profile section is 

specified with the argument –profile. The configuration file that contains the profile is specified with the command 

line argument –settings. 

See the following samples for profile sections that correspond to the above configuration. 

The configuration file is called ftp_settings.ini and it contains the profile ftp_8of9: 

… 

[ftp_8of9] 

protocol = ftp 

host = 8of9 

user = sos 



password = sos 

remote_dir = \remote\sosftp\inbound\ 

… 

Example 1 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_8of9 -operation=send 

-local_dir=samples\outbound\ a1.xml b1.xml c1.xml 

Example 2 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_8of9 -operation=send 

-local_dir=samples\outbound\ -file_spec=.* 

4.2 Simple Transfer 

Here are some example script calls of the SOSFTP client with the Windows SOSFTP start script. 

A complete description of the available parameters can be found in the next chapter. 

Sending Files by FTP 

The used profile of the configuration file ftp_settings.ini: 

… 

[ftp_send] 


host = 8of9 

user = sos 


file_spec = .* 

local_dir = outbound/ 

remote_dir = /inbound/ 

… 

Command 1: 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_send -operation=send 

All files of the local directory outbound/ are sent to the remote directory /inbound/. 

The next three commands have the same result. The files a2.xml, b2.xml and c2.xml of the local directory 

outbound/ are send to the remote directory /inbound/. 



For Unix the files separated by a semicolon ‘;’ have to be quoted with quotation marks ‘”’. The directly listed 

files gets a higher priority than the file_spec parameter. 

Command 2: 


-local_dir=outbound\ a2.xml;b2.xml;c2.xml 

Command 3: 


-local_dir=outbound\ a2.xml b2.xml c2.xml 

Command 4: 


-local_dir= file_path=outbound\a2.xml;outbound\b2.xml;outbound\c2.xml 

Receiving Files by FTP 


… 

[ftp_receive] 


host = 8of9 

user = sos 



local_dir = inbound/ 

remote_dir = /outbound/ 

… 

Command 5: 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_receive -operation=receive 

All files from the remote directory /outbound/ are downloaded to the local directory inbound/. 

The next three Commands have the same result. The files a2.xml, b2.xml and c2.xml of the remote directory 

/outbound/ are downloaded to the local directory inbound/. 

At Unix the files separated by a semicolon ‘;’ have to be quoted with quotation marks ‘”’. The directly listed files 

gets a higher priority than the file_spec parameter. 

Command 6: 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_receive -operation=receive - 

remote_dir=/outbound/ a2.xml;b2.xml;c2.xml 



Command 7: 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_receive -operation=receive 

/outbound/a2.xml /outbound/b2.xml /outbound/c2.xml 

Command 8: 

sosftp.cmd -settings=ftp_settings.ini -profile=ftp_receive -operation=receive - 

remote_dir= outbound/a2.xml;/outbound/b2.xml;/outbound/c2.xml 

Sending Files by SFTP with Password Authentication 


… 

[sftp_send] 

protocol = sftp 

host = kyrill 

user = sos 


ssh_auth_method = password 



remote_dir = /home/sos/inbound/ 

… 

Command 9: 

sosftp.cmd -settings=ftp_settings.ini -profile=sftp_send -operation=send 

All files of the local directory outbound/ are send to the remote directory /home/sos/inbound/. 

Receiving Files by SFTP with Password Authentication 


… 

[sftp_kyrill_receive] 


host = kyrill 

user = sos 





remote_dir = /home/sos/outbound/ 

… 



Command 10: 

sosftp.cmd -settings=ftp_settings.ini -profile=sftp_receive -operation=receive 

All files of the remote directory /home/sos/outbound/ are downloaded to the local directory inbound/. 

Sending Files by SFTP with Public Key Authentication 

The profile used in the configuration file ftp_settings.ini is: 

… 

[sftp_pki_send] 


host = kyrill 

user = sos 

ssh_auth_method = publickey 

ssh_auth_file = ssh/mykey_rsa 



remote_dir = /home/sos/inbound/ 

… 

Command 11: 

sosftp.cmd -settings=ftp_settings.ini -profile=sftp_pki_send -operation=send 

All files of the local directory outbound/ will be sent to the remote directory /home/sos/inbound/. 

Instead of password authentication the private key contained in the file mykey_rsa in the subdirectory ssh/ is 

used for authentication. The corresponding public key has to be installed before on the host kyrill. 

Receiving Files by SFTP with Public-Key Authentication 

The profile in use is contained in the configuration file ftp_settings.ini: 

… 

[sftp_pki_receive] 


host = kyrill 

user = sos 

ssh_auth_method = publickey 

ssh_auth_file = ssh/mykey_rsa 



remote_dir = /home/sos/outbound/ 

… 



Command 12: 

sosftp.cmd -settings=ftp_settings.ini -profile=sftp_pki_receive -operation=receive 

All files of the remote directory /home/sos/outbound/ are transferred to the local directory inbound/. 

Instead of password authentication the private key contained in the file mykey_rsa in the subdirectory ssh/ is 

used for authentication. The corresponding public key has to be installed on the host kyrill beforehand. 

4.3 Install Operation 

Before using the Jump Transfer the SOSFTP Client has to be installed for the designated Jump Host. 

Installation of SOSFTP for the Jump Host 

The profile of the configuration file ftp_settings.ini: 

… 

[install_kyrill] 


host = kyrill 

user = sos 



make_dirs = yes 

classpath_base = /home/sos/sosftp_installation 

… 

Command: 

sosftp.cmd -settings=ftp_settings.ini -profile=install_kyrill -operation=install 

Installs the SOSFTP Client in the directory /home/sos/sosftp_installation at the host kyrill. Should 

the directory /home/sos/sosftp_installation not exist then it will be created automatically. The host 

kyrill does serve as a Jump Host for Jump Transfer execution. Remote Execution does not require such an 

installation. The new installation will contain the files of the default package SOSFTP Client. Other files that has 

been added by the user are not included in the new installation. 



4.4 Jump Transfer 

Remote Execution (execute jump-command) 


… 

[remote_exe] 

;jump_protocol = sftp (default) 

jump_host = kyrill 

jump_user = sos 

jump_password = sos 

jump_ssh_auth_method = password 

jump_command = touch /home/sos/testfile 

[remote_exe_pki] 




jump_ssh_auth_method = publickey 

jump_ssh_auth_file = ssh/mykey_rsa 

jump_command = touch /home/sos/testfile 

… 

Command 1: 

sosftp.cmd -settings=ftp_settings.ini -profile=remote_exe -operation=execute 

Command 2: 

sosftp.cmd -settings=ftp_settings.ini -profile=remote_exe_pki -operation=execute 

Both commands configure the command “touch /home/sos/testfile” to be executed on the host kyrill. 

A file with the name testfile will be created in the directory /home/sos/. 

Remote Sending with FTP, Copy and Move 


… 

[jump_kyrill_and_send_to_8of9] 






jump_command = /home/sos/sosftp_installation/sosftp.sh 





host = 8of9 

user = sos 




… 

Command 3: Copy Files 

sosftp.cmd -settings=ftp_settings.ini -profile=jump_kyrill_and_send_to_8of9 

-operation=send 

All files of the local directory outbound/ will be sent to the Jump Host kyrill and then will be forwarded by 

FTP to the remote directory /inbound/ at the host 8of9. For the jump_command the SOSFTP Client script of 

the installation is assigned. Due to the fact that the host kyrill is running a Unix operating system the shell 

version of the client script is used. 

Command 4: Move Files 

sosftp.cmd -settings=ftp_settings.ini -profile=jump_kyrill_and_send_to_8of9 

-operation=send -remove_files=yes 

All files of the local directory outbound/ will be sent to the Jump Host kyrill and then will be forwarded by 

FTP to the remote directory /inbound/ at the host 8of9. After successful transfer the files that have been 

transferred will be removed from the directory of the local host. 

Remote Receiving with FTP, Copy and Move 


… 

[jump_kyrill_and_receive_from_8of9] 






jump_command = /home/sos/sosftp_installation/sosftp.sh 



host = 8of9 

user = sos 






… 

Command 5: Copy Files 

sosftp.cmd -settings=ftp_settings.ini -profile=jump_kyrill_and_receive_from_8of9 

-operation=receive 

All files of the remote directory /outbound/ of the host 8of9 are downloaded by FTP to the Jump Host 

kyrill and from there are forwarded to the local directory inbound/ at the localhost. For the jump_command 

the SOSFTP client script of the installation is declared. Due to the fact that the host kyrill is running Unix the 

shell version of the client script is used. 

Command 6: Move Files 

sosftp.cmd -settings=ftp_settings.ini -profile=jump_kyrill_and_receive_from_8of9 

-operation=receive -remove_files=yes 

All files of the remote directory /outbound/ of the host 8of9 are downloaded by FTP to the Jump Host 

kyrill and from there are forwarded to the local directory inbound/ at the localhost. After successfully copying 

the previously downloaded files in the remote directory of the remote host 8of9 are deleted. 



4.5 Polling for files 

(receive only) 

The previous use cases for file transfer require that the files that should be transferred do exist in the source 

directory before the SOSFTP Client starts. The polling-feature allows that the files appear in the source directory 

after the SOSFTP client is started. Configured with the parameter ”poll_timeout“ the client will not terminate 

until the specified timeout is reached or a file is found in the source directory during a file check which is performed 

in the interval specified by the parameter “poll_interval”. 

Example Command: 

sosftp.cmd -settings=ftp_settings.ini -profile=polling_receive -operation=receive 


… 

[polling_receive] 


host = 8of9 

user = sos 





poll_timeout = 30 

poll_interval = 60 

… 

For 30 minutes every 60 seconds the SOSFTP Client will check if there are any files (.*) in the remote directory. 

As soon as at least one file is found during this check all files in the remote directory are downloaded into the 

local directory and the client terminates with a success message. 

After 30 minutes if no files are found in the remote directory the client terminates with the error “no matching 

files”. 

The additional parameter “poll_minfiles” allows a regulation of the file transfer by a threshold N in respect to 

the number of files in the remote directory. The file transfer starts only if at least N files are found in the remote 

directory. 



4.6 Logging 

There are two options for logging, the transfer log and the transfer history. Both can be used independently from 

each other: 

4.6.1 Transfer Log 

The transfer log contains all relevant transfer information. By default this is logged to stdout. Instead of logging to 

stdout you can redirect the log output into a file by using the parameter “log_filename”. Should this file not yet 

exist then it will be created otherwise the log output is appended to the existing file. 

4.6.2 Transfer History 

(send and receive only, not for jump) 

The transfer history is created in order to store the file transfer information of the SOSFTP Client into a CSV file 

and subsequently into a database. The information about file transfers is required for further processing steps 

like automated alerting. By default this log type is not active as it requires the SOSFTP server to be in place. 

Example Command 1: 



… 

[ftp_send] 


host = 8of9 

user = sos 





history = history.csv 

… 

By setting a filename for the parameter “history” this log output is written to that file in CSV format. Should the 

file not yet exist then it will be created otherwise the log output will be appended to the existing file. The transfer 

history is one text line per transferred file. If one call of the SOSFTP Client causes the transfer of N files then N 

records are created. The transfer history is only created for the operations send and receive. Each record contains 

the following values: 

guid;mandator;transfer_timestamp;pid;ppid;operation; 

localhost;localhost_ip;local_user;remote_host;remote_host_ip;remote_user; 

protocol;port;local_dir;remote_dir;local_filename;remote_filename;file_size; 

md5;status;last_error_message;log_filename 



Example record of one transferred file: 

1235129738140;SOS;2009-02-19 12:35:38;0;0;send; 

beh;192.11.0.33;sos;wilma;192.11.0.61;test; 

ftp;21;C:\sosftp\out;/tmp/inbound/;1.txt;1.txt;1509; 

939af88ca294a5a74d04036bd5955599;success;; 

The file can be parsed by a SOSFTP Server for importing the new record sets into the Transfer History Database. 

To avoid duplicate entries in the database identical record sets will be detected and only new record sets 

will be imported. 

Example Command 2: 



… 

[ftp_send] 


host = 8of9 

user = sos 





scheduler_host = 8of9 

scheduler_port = 4994 

… 

By setting the parameters “scheduler_host” and “scheduler_port” the SOSFTP Client sends each history 

record set to a Job Scheduler as an UDP message. Such an UDP message can be accepted as an order if the 

Job Scheduler is configured as SOSFTP Server. In this case, the message data will be instantly imported into 

the Transfer History Database. 

For further details see the parameter reference for history processing. 



4.7 Dynamic Password Retrieval 

Instead of a hard-wired password in your configuration you can define a script. This script has to return the 

password by printing it to stdout. The password script should not write any output to stdout except for the password. 

Sample Dynamic Password Script getpassword.cmd: 

@echo sos 

You include this script by just setting it as password. Embedded into backticks “`” the password is interpreted as 

a script which is evaluated: 

password = `getpassword.cmd` 

If the exit code of that script is 0 then its output to stdout is interpreted as a password. If the exit code of that 

script is not 0 then the original parameter value is used as a password. For security concerns the script output 

will not get logged - neither for stdout nor for stderr. Should you run into trouble then you could increase the log 

level to level 3 in order to see what script is executed and to level 8 in order to check if the exit code is 0. 

Sending Files by FTP authenticated with a Dynamic Password 


… 

[ftp_send_dyn] 


host = 8of9 

user = sos 

; password = sos ; hard wired password 

password = `.\getpassword.cmd` ; dynamic password 




… 


sosftp.cmd -settings=ftp_settings.ini -profile=ftp_send_dyn -operation=send 

Dynamic Password Scripts can be used alternatively to the parameter “password” for the protocols FTP, SFTP 

and FTPS. The parameter “jump_password” for the Jump Host does NOT support Dynamic Password Scripts. 

The path of the Dynamic Password Script is always relative to your local host. 



5 Use Cases Server 

The configuration of the SOSFTP Server can be found in the scheduler_sosftp_history.xml and 

scheduler_sosftp_history_params.xml files. Once you have followed the above installation instructions 

you will find these files in the Job Scheduler configuration directory [install_path]/config/ 

where [install_path] is the installation path of the Job Scheduler. 

5.1 History Import 

Figure 9: Jobs and Job Chains for importing the File Transfer History 

scheduler_sosftp_history.xml 

This file contains the following job chains and jobs which support the history import: 

For Import by Receiving Remote Files: 

Job chain scheduler_sosftp_history_receive (Job Chain 1) 

Job chain scheduler_sosftp_history_file_order (Job Chain 2) 

Job scheduler_sosftp_history_receive (Collect Job) 

Job scheduler_sosftp_import (Import Job) 

For Import by Receiving UDP Messages: 

Job chain scheduler_sosftp_history (Job Chain 3) 

Job scheduler_sosftp_import (Import Job) 

For both variants the same Import Job is used. 



scheduler_sosftp_history_params.xml 

This file contains the parameters for the JDBC database connection used for import of the transfer history file: 

 

 

 

 

 

 

 

 

This is a sample configuration for an Oracle Database. 

For further information about the configuration of database connections see the Job Scheduler Reference 

Documentation: 

http://www.sos-berlin.com/doc/en/scheduler.pdf 

5.1.1 Job Configuration 

Here you will find sample configurations for collecting and importing transfer history files: 

scheduler_sosftp_history_receive (Job Chain 1): 

 

 

 

 

 

scheduler_sosftp_history_receive (Collect Job): 

This job copies the history file (csv) of one client host to a directory monitored by the SOSFTP Server. 


order = "yes" 

process_class = "multi"> 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

scheduler_sosftp_history_file_order (Job Chain 2): 

 

 

 

 

 



scheduler_sosftp_import (Import Job): 

Import the history record sets into the database. 

 

 

 

 

 

 

 

 

 

 

scheduler_sosftp_history (Job Chain 3): 

 

 

 

 

 



5.1.2 Setting up orders for receiving history files 

For each SOSFTP Client you need an individual order to receive its history file(s). This order is sent by a Job 

Scheduler command. The order contains the parameters for the job scheduler_sosftp_history_receive. 

For parameter reference see the job documentation of the job JobSchedulerFTPReceive at http://www.sosberlin.com/download/scheduler/sources/sos/scheduler/jobdoc/JobSchedulerFTPReceive.xml. 

Sample order: 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The SOSFTP Server executes this command at the last day of each month at 10 o’clock. If you wish to receive 

the history files from another SOSFTP Client then just add another element to and 

adjust the values accordingly. 



5.2 Custom Fields 

The Custom Fields provide an additional feature for the transfer history. You can define your own parameters. 

These parameters have to be set at your SOSFTP Client call. They are written into the transfer history file, imported 

into your database and can be checked in the Web GUI and/or processed by the Reporting Jobs. 

5.2.1 SOSFTP Client 

The SOSFTP Client can process your own parameters if you write them at the command line in the form 

-history_entry_fieldname=fieldvalue 

The transfer history will contain a new field fieldname and the corresponding data sets will additionally contain 

the value fieldvalue. The Import Job of the SOSFTP Server will automatically import the custom fields into 

your database. 


sosftp.cmd … -history_entry_myfield1=value1 –history_entry_myfield2=value2 … 

After executing this command the transfer history file will contain two new column names myfield1 and 

myfield2: 

…;md5;status;last_error_message;log_filename;myfield1;myfield2 

The additional values are appended to the corresponding data sets: 

…;939af88ca294a5a74d04036bd5955599;success;;;value1;value2 



5.2.2 Web GUI 

If your database contains your own parameters then you are able to watch them in the Web GUI. 

Figure 10: The Custom Fields “custom_field_1” and “custom_field_2” in the Web GUI 

To activate the view for the new parameters you have to adapt the PHP-Array $sosftp_custom_fields in 

your Web GUI configuration file sosftp_web/custom/custom.inc.php. 

Sample Configuration for Figure 9: 

/* 

-- array key : database field name 

*/ 

$sosftp_custom_fields = array( 

'CUSTOM_FIELD_1' => '', 

'CUSTOM_FIELD_2' => '', 

); 

This sample configuration expands the view of the Web GUI by the Custom Fields columns CUSTOM_FIELD_1 

and CUSTOM_FIELD_2 of the database table SOSFTP_FILES_HISTORY. 



Figure 11: The Custom Fields “custom_field_1” and “custom_field_2” are titled “Title_1” and “Title_2”. The value 

of “custom_field_2” is also displayed in the fields of column “Title_1”. 

Sample Configuration for Figure 10: 

/* 

-- array key : database field name 

-- optional 

-- title : field title as th (default - field name) 

-- width : td width (default 50) 

-- type : type for sorting (string, numeric: default - string) 

-- fields: field values will be displayed in the same column (separated by ) 

*/ 

$sosftp_custom_fields = array( 

'CUSTOM_FIELD_1' => array('title'=>'CUSTOM_FIELD_1 Title', 'width'=>'100', 

'type'=>'string', 'fields'=>'CUSTOM_FIELD_2'), 

); 

'CUSTOM_FIELD_2'=>array('title'=>'CUSTOM_FIELD_2 Title', 'width'=>'100', 

'type'=>'string'), 



This sample configuration sets individual titles for the two Custom Fields in the table header. Additionally the 

values of CUSTOM_FIELD_2 are embedded in the column of CUSTOM_FIELD_1. 


This is a step by step guide to get your first SOSFTP Report. 

5.3.1 Configuration 

Here are examples for two job configurations of the two JasperReports jobs defined in the file 

[install_path]/config/scheduler_jasperreports_sosftp.xml. 

This is the default configuration. 

Example 1) 

The following job generates a report with information targeted at Application Managers on the first day on every 

month. 

 

 

 

 

 

 

 

 

 

 

 


language="java"/> 

 

 

 

 

 

 

Example 2) 

The following job generates a report with customer information on the first day of every month for each mandator. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 



 

 

 

 

 

The job parameter repeat_sql_statement accepts a SQL string as well as a path to a SQL script. 

In Example 2 the path to a SQL script is given. Short SQL statements can also be declared directly: 

 

Database Connection 

Customize the file [scheduler install_path]/jasperreports/config/sos_settings_sosftp.ini 

Example settings for an Oracle database: 

[configuration] 

class = SOSOracleConnection 

driver = oracle.jdbc.driver.OracleDriver 

url = jdbc:oracle:thin:@8of9:1521:test 

user = test 

password = test 

compatibility = normal 

5.3.2 Generating a Report 

Start the SOSFTP Server and run the job jasper_report_sosftp. The job creates in the directory 

[scheduler install_path]/jasperreports/reports/ the files sosftp.pdf and sosftp.xls. 





Figure 12: Sample sosftp.pdf. A SOSFTP Report in PDF format for the Application Manager containing 

the transfer information of three mandators. 



6 Complete Client Parameter Reference 

operation Default: --- 

settings Default: --- 


send, receive, remove, execute or install. 

send - transfer files by FTP/SFTP to a remote server 

receive - transfer files by FTP/SFTP from a remote server 

remove - remove files by FTP/SFTP on a remote server 

execute - execute a command by SSH on a remote server 

install - install SOSFTP on a remote server 

[optional] A configuration file can be specified that contains profiles, i.e. sections, 

with parameters specified as pairs of names and values in a plain text 

format like this: 

[sample_transfer] 


host = localhost 

port = 21 

local_dir = /tmp 

... 

profile Default: --- 

At the command line the name of the configuration file and the profile 

are specified like this: 

sosftp.sh -operation=send -settings=settings.ini - 

profile=sample_transfer ... 

A profile can reference the contents of other profiles like this: 

[default] 

history = /sosftp/transfer_history.csv 

mandator = SOS 

scheduler_host = localhost 

scheduler_port = 4444 

[sample_transfer] 

include = default 


host = www.sos-berlin.com 

port = 21 

local_dir = /tmp 

... 

With this sample the profile sample_transfer includes the default profile 

via the include directive and thus applies the file transfer history settings. 

[optional] If a configuration file is being used (see parameter settings), then this


verbose Default: --- 


parameter specifies a section within the configuration file. Such sections, 

i.e. profiles, specify parameters as pairs of names and values that 

otherwise would be specified by command line parameters. 

At the command line the name of the configuration file and the profile 

are specified like this: 

sosftp.sh -operation=send -settings=settings.ini - 

profile=sample_transfer ... 

Parameters for logging 

[optional] The verbosity level specifies the intensity of log entries. A value between 

1 and 9 can be specified. Higher values cause more detailed information 

to be logged. 

log_filename Default: --- 

Log output is written to stdout or to a file that has been specified with 

the parameter log_filename. 

[optional] This parameter specifies the location of a file to which the log output 

will be written. 

banner_header Default: --- 

Should the file not exist then it will be created. If the already exists 

then all log output will be appended. 

Without specifying this parameter all log output will be written to 

stdout. 

[optional] This program logs output to stdout or to a file that has been specified 

by the parameter log_filename. A template can be used in order to 

organize the output that is created. The output is grouped into header, 

file list and footer. 

This parameter specifies a template file for header output. 

Templates can use internal variables and parameters as placeholders in 

the form %{placeholder}. The standard header template looks like this: 

************************************************************ 

************* 

* 

* 

* SOSFTP - Managed File Transfer Utility 

* 

* -------------------------------------- 

* 

* 

* 

************************************************************ 

************* 

version = %{version}


banner_footer Default: --- 


date = %{date} %{time} 

operation = %{operation} 

protocol = %{protocol} 

file specification = %{file_spec} 

file path = %{file_path} 

source host = %{localhost} (%{local_host_ip}) 

local directory = %{local_dir} 

jump host = %{jump_host} 

target host = %{host} (%{host_ip}) 

target directory = %{remote_dir} 

pid = %{current_pid} 

ppid = %{ppid} 

************************************************************ 

************* 

[optional] This program logs output to stdout or to a file that has been specified 

by the parameter log_filename. A template can be used in order to 

organize the output that is created. The output is grouped into header, 

file list and footer. 

protocol Default: ftp 

host Default: --- 

port Default: 21 

This parameter specifies a template file for footer output. 

Templates can use internal variables and parameters as placeholders in 

the form %{placeholder}. The standard footer template looks like this: 

************************************************************ 

************* 

execution status = %{status} 

successful transfers = %{successful_transfers} 

failed transfers = %{failed_transfers} 

last error = %{last_error} 

************************************************************ 

************* 

Parameters for server connections 

user Default: anonymous 

password Default: --- 

The values ftp, sftp or ftps are valid for this parameter. 

If sftp is used, then the ssh_* parameters will be applied. 

Host or IP address from which or to which files should be transferred. 

Port by which files should be transferred. For FTP this is usually port 21, 

for SFTP this is usually port 22. 

User name for authentication at the FTP/SFTP server.


[optional] Password for authentication at the FTP/SFTP server. 

account Default: --- 


For SSH/SFTP connections that make use of public/private key authentication 

the password parameter is specified for the passphrase that 

optionally secures a private key. 

[optional] Optional account info for authentication with an FTP server. 

transfer_mode Default: binary 

[optional] Transfer mode is used for FTP exclusively and can be either ascii or 

binary. 

passive_mode Default: 0 

[optional] Passive mode for FTP is often used with firewalls. Valid values are 0 or 

1. 

remote_dir Default: . 

local_dir Default: . 

file_spec Default: .* 

Directory at the FTP/SFTP server from which or to which files should be 

transferred. 

By default the home directory of the user at the FTP/SFTP server is 

used. 

Local directory into which or from which files should be transferred. By 

default the current working directory is used. 

Besides paths in the local file system UNC path names are supported 

that could be used to address remote server systems: 

\\somehost\somedirectory can be used in the same way as 

//somehost/somedirectory to transfer files from an FTP/SFTP server 

to a different remote server system. 

Moreover, you could specify URIs for a file schema as in 

file:////somehost/somedirectory. Please, consider the required 

number of slashes. file URIs are subject to the following limitations 

due to constraints of the underlying Java JRE: 

File names and path names must not contain any spaces. 

Authentication by authority strings as in 

file:////user:password@somehost/somedirectory is not supported. 

[optional] This parameter expects a regular expression to select files from a local 

directory or from an FTP/SFTP server (depending on the operation parameter 

values send or receive) to be transferred. 

file_path Default: --- 

For the operations send and receive either this parameter has to be 

specified or the parameter file_path or a list of file names as additional 

parameters.


[optional] This parameter is used alternatively to the parameter file_spec to 

specify a single file for transfer. 


When receiving files the following applies: 

This parameter accepts the absolute name and path of file at the 

FTP/SFTP server that should be transferred. The file name has to include 

both name and path of the file at the FTP/SFTP server. 

The file will be stored unter its name in the directory that is specified by 

the parameter local_dir. 

The following parameters are ignored should this parameter be used: 

file_spec and remote_dir. 

When sending files the following applies: 

This parameter accepts the absolute name and path of file that should 

be transferred. An absolute path has to be specified. 

The file will be stored under its name in the directory at the FTP/SFTP 

server that has been specified by the parameter remote_dir. 

The following parameters are ignored should this parameter be used: 

file_spec and local_dir. 

File set operations 

file_spec2 Default: --- 

[optional] In addition to what is stated for the parameter file_spec additional 

parameters can be specified for up to 9 file sets like this: 

-file_spec=.*\.gif$ -local_dir=/tmp/1 -remote_dir=/tmp/1 

-file_spec2=.*\.exe$::param_set_2 - 

param_set_2="transfer_mode=binary::remote_dir=/tmp/2::local_di 

r=/tmp/2" 

Within the file_spec2 parameter value the regular expression is separated 

by :: from the name of a file set. This name can freely be chosen, 

it consists of the characters 0-9, a-z and _. 

The name of the file set is used as a separate parameter in the command 

line. This parameter is assigend the list of parameters that should 

be valid for the specific file set. Therefore the names and values of individual 

parameters are specified in the form name=value::name2=value2 

.... Such parameters are exclusively valid for the specific file set. 

The above sample causes all files with the extension .gif to be transferred 

from the local directory /tmp/1 to a directory with the same


transactional Default: false 


name on the target host. For files with the extension .exe a file set param_set_2 

is specified that contains parameters that are specific for 

this file set, as binary transfer and different source and target directories. 

Please, consider that parameter file sets cannot specify parameters that 

control the connection to a target host, i.e. all files are transferred between 

the same local and remote hosts. However, the transfer direction 

can be changed, e.g. by specifying a different operation parameter for 

a file set. 

Optional parameters for transfer control 

[optional] This parameter specifies if file transfers should be operated within a 

single transaction, i.e. either all files are successfully transferred or 

none. Should an error occur during a transfer operation then all transfers 

will be rolled back. 

atomic_suffix Default: --- 

When specifying the value true then the following applies: 

The parameter atomic_suffix has to be specified that causes 

target files to be created with a suffix such as "~" and that causes the 

respective files to be renamed to their target file name after the transfer 

of all files has been successfully completed. If at least one file out of 

a set of files cannot be transferred successfully then no files will be renamed, 

instead the temporarily created files are removed from the target 

system. 

The parameter remove_files that causes files to be removed 

after successful transfer will be effective only after all files have been 

successfully transferred. Otherwise no files will be removed. 

[optional] This parameter specifies whether target files should be created with a 

suffix such as "~", and should be renamed to the target file name after 

the file transfer is completed. This mechanism is useful if the target 

directory is monitored for incoming files by some application and if files 

are required to appear atomically instead of being subsequently written 

to. 

remove_files Default: false 

The temporary suffix is specified as the value of this parameter. 

This setting is recommended should target directories be monitored by 

an application or a Job Scheduler. 

[optional] This parameter specifies whether files on the FTP/SFTP server should be 

removed after transfer. 

skip_transfer Default: false


[optional] If this Parameter is set to true then all operations except for the transfer 

itself will be performed. This can be used to just trigger for files or to 

only delete files on the FTP/SFTP server. 

overwrite_files Default: true 

[optional] This parameter specifies if existing files should be overwritten. 

append_files Default: false 


Should this parameter be used with force_files und should no files be 

transferred due to overwrite protection then an error will be raised stating 

that "no matching files" could be found. 

[optional] This parameter specifies whether the content of a source file should be 

appended to the target file should this file exist. 

force_files Default: true 

The parameter overwrite_files will be ignored if this parameter is 

specified with the value true. 

[optional] This parameter specifies whether an error should be raised if no files 

could be found for transfer. 

zero_byte_transfer Default: yes 

The number of files to be transferred is determined by the file_spec or 

file_path parameters and can be restricted by the overwrite_files 

parameter should this be specified with the value false. 

[optional] This parameter specifies whether zero byte files should be transferred 

and processed by subsequent commands. The following settings are 

available: 

recursive Default: false 

yes : Files with zero byte size are transferred (default). 

no : Files with zero byte size are transferred, should at least one 

of the files have more than zero byte size. 

strict : Files with zero byte size are not transferred. An error will 

be raised if any zero byte file is found. 

relaxed : Files with zero byte size will not be transferred. 

However, no error will be raised if this results in no files being transferred. 

Use of this parameter can be refined using the force_files parameter: 

should force_files have the value false, then processing will be 

treated as successful in the event of no files having been transferred. 

Note that the remove_files parameter has unrestricted validity. Files 

with zero byte size will be removed regardless of whether or not they 

have been transferred. 

[optional] This parameter specifies if files from subdirectories should be transferred 

recursively. Recursive processing is specified by one of the values 

yes or true.


check_size Default: true 


Regular expression matches apply to files from subdirectories as specified 

by the parameter file_spec. 

Parameters for transfer verification 

[optional] This parameter determines whether the original file size and the number 

of bytes transferred should be compared after a file transfer and 

whether an error should be raised if they would not match. 

check_retry Default: 0 

[optional] This parameter specifies whether a file transfer should be repeated in 

order to ensure that the file was complete when the transfer started. 

This is relevant for Unix systems that allow read and write access to a 

file at the same time. 

check_interval Default: 60 

This parameter causes the size of the current file transfer and of the 

previous file transfer to be compared and repeats transferring one file 

up to the number of trials specified by this parameter. Should the file 

size of both transfers be the same, then it is assumed that the file was 

complete at the FTP/SFTP server. 

The interval between two trials to transfer a file is configured using the 

check_interval parameter. 

[optional] This parameter specifies the interval in seconds between two file transfer 

trials, if repeated transfer of files has been configured using the 

check_retry parameter. 

poll_timeout Default: 0 

Parameters for polling 

[optional] This parameter specifies the time in minutes, how long a file is polled 

for. 

poll_interval Default: 60 

If a file becomes available within the time specified then it will be transferred, 

otherwise an error "no matching files" will be raised. 

[optional] This parameter specifies the interval in seconds, how often a file is 

polled for within the duration that is specified by the parameter 

poll_timeout. 

poll_minfiles Default: 1 

[optional] This parameter specifies the number of files that have to be found during 

the polling period in order to cause the transfer to start. This pa-


replacing Default: --- 


rameter is used exclusively with the parameters poll_timeout. 

Parameters for file processing 

[optional] Regular expression for filename replacement with the value of the parameter 

replacement. 

replacement Default: --- 

If the expression matches the filename then the groups found are replaced. 

a) 

For replacement "capturing groups" are used. Only the content of the 

capturing groups is replaced. 

Replacements are separated by a semicolon ";". Example: 

replacing= (1)abc(12)def(.*) 

replacement= A;BB;CCC 

Input file: 1abc12def123.txt 

Output file: AabcBBdefCCC 

b) 

If no "capturing groups" are specified then the entire match is replaced. 

Example: 

replacing= Hello 

replacement= 1234 

Input file: Hello_World.txt 

Output file: 1234_World.txt 

Requires the parameter replacement to be specified. 

[optional] String for replacement of matching character sequences within file 

names that are specified with the value of the parameter replacing. 

If multiple "capturing groups" shall be replaced then one replacement 

string per group has to be specified. These strings are separated by a 

semicolon ";": 

replacement= aa;[filename:];bb 

Supports masks for substitution in the file name with format strings that 

are enclosed with [ and ] . The following format strings are supported: 

[date: date format ] 

date format must be a valid Java data format string, e.g. 

yyyyMMddHHmmss , yyyy-MM-dd.HHmmss etc. 

[filename:] 

Will be substituted by the original file name. 

[filename:lowercase]


root Default: --- 


Will be substituted by the original file name including the file extension 

with all characters converted to lower case. 

[filename:uppercase] 

Will be substituted by the original file name including the file extension 

with all characters converted to upper case. 

Requires the parameter replacing to be specified. 

[optional] The parameter specifies the directory in which this program is allowed 

to create temporary files. Temporary files are required if due to the parameter 

setting jump_host files have to be stored on an intermediary 

server and will be removed after completion of the transfer. 

alternative_host Default: --- 

Without this parameter the temporary directory is used as provided by 

the operating system. 

Parameters for alternative connections 

[optional] Alternative parameter for the primary parameter host. 

alternative_port Default: --- 

The alternative parameters are used solely should the connection to an 

FTP/SFTP server fail, e.g. if the server were not available or if invalid 

credentials were used. 

[optional] Alternative parameter for the primary parameter port. 

alternative_user Default: --- 




[optional] Alternative parameter for the primary parameter user. 

alternative_password Default: --- 




[optional] Alternative parameter for the primary parameter password. 

alternative_account Default: --- 




[optional] Alternative parameter for the primary parameter account.


alternative_remote_dir Default: --- 





[optional] Alternative parameter for the primary parameter remote_dir. 

alternative_passive_mode 




Default: --- 

[optional] Alternative parameter for the primary parameter passive_mode. 

alternative_transfer_mode 




Default: --- 

[optional] Alternative parameter for the primary parameter transfer_mode. 

ssh_proxy_host Default: --- 




Parameters for SSH connections 

[optional] The value of this parameter is the host name or the IP address of a 

proxy that is used in order to establish a connection to the SSH server. 

The use of a proxy is optional. 

ssh_proxy_port Default: --- 

[optional] This parameter specifies the port number of the proxy, should a proxy 

be used in order to establish a connection to the SSH server. 

ssh_proxy_user Default: --- 

[optional] The value of this parameter specifies the user account for authentication 

by the proxy server should a proxy be used in order to connect to 

the SSH server. 

ssh_proxy_password Default: --- 

[optional] This parameter specifies the password for the proxy server user account, 

should a proxy be used in order to connect to the SSH server. 

ssh_auth_method Default: publickey 

[optional] This parameter specifies the authentication method for the SSH server - 

the publickey and password methods are supported.


ssh_auth_file Default: --- 


Should the publickey authentication method be used, then the path 

name of the private key file has to be specified with the ssh_auth_file 

parameter. Should the private key file be secured by a passphrase then 

the passphrase has to be specified with the password parameter. 

For the password authentication method the password for the user account 

has to be specified using the password parameter. 

The authentication methods that are enabled depend on the SSH server 

configuration. Not all SSH servers are configured for password authentication. 

[optional] This parameter specifies the path and name of a user's private key file 

that is used for authentication with an SSH server. 

http_proxy_host Default: --- 

This parameter has to be specified should the publickey authentication 

method have been specified in the ssh_auth_method parameter. 

Should the private key file be secured by a passphrase, then the 

passphrase has to be specified using the password parameter. 

Parameters for SSL/TLS connections 


proxy used in order to establish a connection to the SSH server via 

SSL/TLS. The use of a proxy is optional and exclusively considered if 

the parameter protocol=ftps is used. 

http_proxy_port Default: --- 

[optional] This parameter specifies the port of a proxy that is used in order to establish 

a connection to the SSH server via SSL/TLS, see parameter 

http_proxy_host. 

jump_protocol Default: sftp 

Parameters used with Jump Hosts 

[optional] When using a jump_host then files are first transferred to this host and 

then to the target system. Different protocols (FTP/SFTP) can be used 

for these transfer operations. 

jump_host Default: --- 

This parameter expects ftp, sftp or ftps to be specified. If sftp is 

used, then the jump_ssh_* parameters will be considered.


[optional] When using a jump_host then files are first transferred to this host and 

then to the target system. Different protocols (FTP/SFTP) can be used 

for these transfer operations. 

jump_port Default: 22 


Host or IP address of the jump_host from which or to which files should 

be transferred in a first operation. 

[optional] Port on the jump_host by which files should be transferred. For FTP this 

is usually port 21, for SFTP this is usually port 22. 

jump_user Default: --- 

[optional] User name for authentication with the jump_host. 

jump_password Default: --- 

[optional] Password for authentication with the jump_host. 

jump_proxy_host Default: --- 


proxy used in order to establish a connection to the jump host. The use 

of a proxy is optional. 

jump_proxy_port Default: --- 

[optional] This parameter specifies the port of a proxy that is used in order to establish 

a connection to the jump host, see parameter jump_proxy_host. 

jump_proxy_user Default: --- 

[optional] The value of this parameter specifies the user account for authentication 

by the proxy server should a proxy be used in order to connect to 

the jump host, see parameter jump_proxy_host. 

jump_proxy_password Default: --- 

[optional] This parameter specifies the password for the proxy server user account, 

should a proxy be used in order to connect to the jump host, see 

parameter jump_proxy_host. 

jump_ssh_auth_method Default: --- 

[optional] This parameter specifies the authentication method for the SSH server - 

the publickey and password methods are supported. 

jump_ssh_auth_file Default: --- 

When the publickey authentication method is used, then the path name 

of the private key file must be set in the jump_ssh_auth_file parameter. 

Should the private key file be secured by a passphrase then this 

passphrase has to be specified by the jump_password parameter. 

For the password authentication method the password for the user account 

has to be specified using the jump_password parameter. 

The authentication methods that are enabled depend on the SSH server 

configuration. Not all SSH servers are configured for password authentication. 

[optional] This parameter specifies the path and name of a user's private key file 

used for login to the SSH server of the jump_host.


jump_command Default: --- 


This parameter must be specified if the publickey authentication method 

has been specified in the jump_ssh_auth_method parameter. 

Should the private key file be secured by a password, then this password 

has to be specified using the jump_password parameter. 

[optional] This parameter specifies a command that is to be executed on the SSH 

server. Multiple commands can be separated by the command delimiter 

that is specified using the jump_command_delimiter parameter. 

jump_command_script Default: --- 

[optional] This parameter can be used as an alternative to jump_command, 

jump_command_delimiter and jump_command_script_file. It contains 

script code which will be transferred to the remote host as a file and will 

then be executed there. 

jump_command_script_f 

ile 

Default: --- 

[optional] This parameter can be used as an alternative to jump_command, 

jump_command_delimiter and jump_command_script. It contains the 

name of a script file, which will be transferred to the remote host and 

will then be executed there. 

jump_command_delimit 

er 

Default: %% 

jump_ignore_error Default: false 

Command delimiter characters are specified using this parameter. 

These delimiters can then be used in the jump_command parameter to 

seperate multiple commands. These commands are then excecuted in 

separate SSH sessions. 

[optional] Should the value true be specified, then execution errors caused by 

commands on the SSH server are ignored. Otherwise such execution 

errors will be reported. 

jump_ignore_signal Default: false 

[optional] Should the value true be specified, then on Unix systems all signals will 

be ignored that terminate the execution of a command on the SSH 

server - if for example a command is terminated using kill. 

jump_ignore_stderr Default: false 

Note that by default errors will be reported for commands that are terminated 

by signals. 

[optional] This job checks if any output to stderr has been created by a command 

that is being executed on the SSH server and reports this as an error. 

jump_simulate_shell Default: false 

Should the value true be specified for this parameter, then output to 

stderr will not be reported as an error by the Job Scheduler. 

[optional] Should the value true be specified for this parameter, then a login to a 

shell is simulated to execute commands. Some scripts may cause prob-


jump_simulate_shell_pro 

mpt_trigger 


lems if no shell is present. 

Default: --- 

[optional] The expected command line prompt. Using this prompt the program 

tries to find out if commands may be entered or have been carried out. 

If no prompt can be configured, then timeout parameters have to be 

used in order to check if the shell is ready to accept commands. 

jump_simulate_shell_log 

in_timeout 

Default: --- 

[optional] If no new characters are written to stdout or stderr after the given 

number of milliseconds, then it is assumed that the login has been carried 

out and that the shell is waiting for the next command. 

jump_simulate_shell_ina 

ctivity_timeout 

Default: --- 

[optional] If no new characters are written to stdout or stderr after the given 

number of milliseconds, then it is assumed that the command has been 

carried out and that the shell is waiting for the next command. 

classpath_base Default: --- 

Parameters for installation 

[optional] The parameter is used during installation of this program on a remote 

server with the parameter operation=install. This parameter specifies 

the path of the Java Runtime Environment (JRE) at the remote 

server and is used if on the remote server a JRE is not included in the 

system path. 

history Default: --- 

The path of the specified JRE is added to the start script at the remote 

server (sosftp.cmd respectively sosftp.sh). 

Parameters for history processing 

[optional] This parameter causes a history file to be written in CSV format. The 

path and name of the history file is specified as value for this parameter. 

A history record is created for each file that has been transferred. 

A history file contains the following columns: 

guid 

A unique identifier for the history entry. This identifier is used for checking 

of duplicate entries in combination with Job Scheduler for Managed 

File Transfer.



mandator 

A character that denominates the mandator of a file transfer, see respective 

parameter. 

transfer_timestamp 

The point in time when the transfer took place. 

pid 

The process id of the current process that executes the file transfer, see 

parameter current_pid. 

ppid 

The process id of the parent of the process that executes the file transfer, 

see respective parameter. 

operation 

One of the operations send or receive, see respective parameter. 

localhost 

The name of the host on which this program is executed. 

localhost_ip 

The IP address of the host on which this program is executed. 

local_user 

The name of the user account for which this program is executed. 

remote_host 

The name of the host to/from which a transfer is executed, see parameter 

host. 

remote_host_ip 

The IP address of the host to/from which a transfer is executed, see 

parameter host. 

remote_user 

The name of the user account for the remote host, see parameter user. 

protocol 

The protocol can be either ftp, sftp or ftps, see respective parameter. 

port 

The port on the remote host, see respective parameter. 

local_dir 

The local directory to/from which a file has been transferred, see respective 

parameter. 

remote_dir 

The remote directory to/from which a file has been transferred, see respective 

parameter. 

local_filename 

For send operations this is the original file name on the local host. 

For receive operations this is the resulting file name on the local host 

optionally having applied replacement operations, see parameter replacing. 

remote_filename


history_repeat Default: 3 


For send operations this is the resulting file name on the remote host 

optionally having applied replacement operations, see parameter replacing. 

For receive operations this is the original file name on the remote host. 

file_size 

The size of the transferred file in bytes. 

md5 

The value of the MD5 hash that is created from the file that was transferred. 

status 

The status can be either success or error. 

last_error_message 

Should an error have occurred then the last error message will be given 

in this column. 

log_filename 

The name of the log file, see respective parameter. 

[optional] The parameter is used in order to synchronize parallel write access to 

the history file by multiple instances of this program. 

history_repeat_interval Default: 1 

This parameter specifies the maximum number of repeat intervals when 

trying to write to the history file if the history file is locked due to parallel 

instances of this program. 

[optional] The parameter is used in order to synchronize parallel write access to 

the history file by multiple instances of this program. 

current_pid Default: --- 

This parameter specifies the the interval in seconds of repeated trials to 

write to the history file if the history file is locked due to parallel instances 

of this program. 

[optional] This parameter is used for Unix systems and - as opposed to other parameters 

- is usually specified in the start script sosftp.sh. The value 

of the environment variable $$ is assigned, that contains the current 

process id (PID). 

ppid Default: --- 

The process id is used when writing an entry to a history file for each 

transfer (see parameter history). 

[optional] This parameter is used for Unix systems and - as opposed to other parameters 

- is usually specified in the start script sosftp.sh. The value 

of the environment variable $PPID is assigned, that contains the process 

id of the current parent process (PPID). 

The parent process id is used when writing an entry to a history file for 

each transfer (see parameter history).


mandator Default: SOS 

[optional] This parameter specifies the mandator for which a file transfer is effected. 

The mandator is added to an optional history file, see parameter 

history and has no technical relevance for the transfer. 

scheduler_host Default: --- 

[optional] This parameter specifies the host name or IP address of a server for 

which Job Scheduler is operated for Managed File Transfer. 

scheduler_port Default: --- 


The contents of an optional history file (see parameter history), is 

added to a central database by Job Scheduler. 

This parameter causes the transfer of the history entries for the current 

transfer by UDP to Job Scheduler. Should Job Scheduler not be accessible 

then no errors are reported, instead, the contents of the history will 

automaticall be processed later on. 

[optional] The port for which a Job Scheduler for Managed File Transfer is operated, 

see parameter scheduler_host. 

scheduler_job_chain Default: scheduler_sosftp_history 

[optional] The name of a job chain for Managed File Transfer with Job Scheduler, 

see parameter scheduler_host. The job chain accepts history entries 

and performs an import into a central database. 

* Default: --- 

Additional parameters 

[optional] Any additional parameters that are not preceeded by keywords represent 

paths for files that should be transferred.

JADE Documentation (PDF) - SOS-Berlin

Create successful ePaper yourself

Delete template?

Save as template?