LibraryToggle FramesPrintFeedback

This component provides access to remote file systems over the FTP and SFTP protocols.

[Tip]Consuming from remote FTP server

Make sure you read the section titled Default when consuming files further below for details related to consuming files.

The options below are exclusive to the FTP2 component:

Name Default Value Description
username null Specifies the username to use to log in to the remote file systen.
password null Specifies the password to use to log in to the remote file system.
binary false Specifies the file transfer mode, BINARY or ASCII. Default is ASCII (false).
disconnect false Camel 2.2: Whether or not to disconnect from remote FTP server right after use. Can be used for both consumer and producer. Disconnect will only disconnect the current connection to the FTP server. If you have a consumer which you want to stop, then you need to stop the consumer/route instead.
localWorkDirectory null When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial, if you consume a very big remote file and thus can conserve memory. See below for more details.
passiveMode false FTP only: Specifies whether to use passive mode connections. Default is active mode {false).
securityProtocol TLS

FTPS only: Sets the underlying security protocol. The following values are defined: TLS: Transport Layer Security SSL: Secure Sockets Layer

disableSecureDataChannelDefaults false Camel 2.4: FTPS only: Whether or not to disable using default values for execPbsz and execProt when using secure data transfer. You can set this option to true if you want to be in absolute full control what the options execPbsz and execProt should be used.
execProt null

Camel 2.4: FTPS only: Will by default use option P if secure data channel defaults hasn't been disabled. Possible values are: C: Clear S: Safe (SSL protocol only) E: Confidential (SSL protocol only) P: Private

execPbsz null Camel 2.4: FTPS only: This option specifies the buffer size of the secure data channel. If option useSecureDataChannel has been enabled and this option has not been explicit set, then value 0 is used.
isImplicit false FTPS only: Sets the security mode(implicit/explicit). Default is explicit (false).
knownHostsFile null SFTP only: Sets the known_hosts file, so that the SFTP endpoint can do host key verification.
privateKeyFilePassphrase null SFTP only: Set the private key file passphrase to that the SFTP endpoint can do private key verification.
privateKeyFilePassphrase null SFTP only: Set the private key file passphrase to that the SFTP endpoint can do private key verification.
strictHostKeyChecking no SFTP only:Camel 2.2: Sets whether to use strict host key checking. Possible values are: no, yes and ask. ask does not make sense to use as Camel cannot answer the question for you as its meant for human intervention. Note: The default in Camel 2.1 and below was ask.
maximumReconnectAttempts 3 Specifies the maximum reconnect attempts Fuse Mediation Router performs when it tries to connect to the remote FTP server. Use 0 to disable this behavior.
reconnectDelay 1000 Delay in millis Fuse Mediation Router will wait before performing a reconnect attempt.
connectTimeout 10000 Camel 2.4: Is the connect timeout in millis. This corresponds to using ftpClient.connectTimeout for the FTP/FTPS. For SFTP this option is also used when attempting to connect.
soTimeout null FTP and FTPS Only:Camel 2.4: Is the SocketOptions.SO_TIMEOUT value in millis. Note SFTP will automatic use the connectTimeout as the soTimeout.
timeout 30000 FTP and FTPS Only:Camel 2.4: Is the data timeout in millis. This corresponds to using ftpClient.dataTimeout for the FTP/FTPS. For SFTP there is no data timeout.
throwExceptionOnConnectFailed false Camel 2.5: Whether or not to thrown an exception if a successful connection and login could not be establish. This allows a custom pollStrategy to deal with the exception, for example to stop the consumer or the likes.
siteCommand null FTP and FTPS Only:Camel 2.5: To execute site commands after successful login. Multiple site commands can be separated using a new line character (\n). Use help site to see which site commands your FTP server supports.
stepwise true When consuming directories, specifies whether or not to use stepwise mode for traversing the directory tree. Stepwise means that it will CD one directory at a time. For more details, see Stepwise changing directories.
separator Auto Camel 2.6: Dictates what path separator char to use when uploading files. Auto means use the path provided without altering it. UNIX means use UNIX style path separators. Windows means use Windows style path separators.
ftpClient null FTP and FTPS Only:Camel 2.1: Allows you to use a custom org.apache.commons.net.ftp.FTPClient instance.
ftpClientConfig null FTP and FTPS Only:Camel 2.1: Allows you to use a custom org.apache.commons.net.ftp.FTPClientConfig instance.
serverAliveInterval 0 SFTP Only:Camel 2.8 Allows you to set the serverAliveInterval of the sftp session
serverAliveCountMax 1 SFTP Only:Camel 2.8 Allows you to set the serverAliveCountMax of the sftp session
ftpClient.trustStore.file null FTPS Only: Sets the trust store file, so that the FTPS client can look up for trusted certificates.
ftpClient.trustStore.type JKS FTPS Only: Sets the trust store type.
ftpClient.trustStore.algorithm SunX509 FTPS Only: Sets the trust store algorithm.
ftpClient.trustStore.password null FTPS Only: Sets the trust store password.
ftpClient.keyStore.file null FTPS Only: Sets the key store file, so that the FTPS client can look up for the private certificate.
ftpClient.keyStore.type JKS FTPS Only: Sets the key store type.
ftpClient.keyStore.algorithm SunX509 FTPS Only: Sets the key store algorithm.
ftpClient.keyStore.password null FTPS Only: Sets the key store password.
ftpClient.keyStore.keyPassword null FTPS Only: Sets the private key password.
[Important]FTPS component default trust store

By default, the FTPS component trust store accept all certificates. If you only want trust selective certificates, you have to configure the trust store with the ftpClient.trustStore.xxx options or by configuring a custom ftpClient.

[Important]More options

See File2 for more options, as all the options from File2 are inherited by FTP2.

You can configure additional options on the ftpClient and ftpClientConfig from the URI directly by using the ftpClient. or ftpClientConfig. prefix.

For example to set the setDataTimeout on the FTPClient to 30 seconds you can do:

from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000")
    .to("bean:foo");

You can mix and match and have use both prefixes, for example to configure date format or timezones.

from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000&ftpClientConfig.serverLanguageCode=fr")
    .to("bean:foo");

You can have as many of these options as you like.

See the documentation of the Apache Commons FTP FTPClientConfig for possible options and more details. And as well for Apache Commons FTP FTPClient.

If you do not like having many and long configuration in the url you can refer to the ftpClient or ftpClientConfig to use by letting Camel lookup in the Registry for it.

For example:

   <bean id="myConfig" class="org.apache.commons.net.ftp.FTPClientConfig">
       <property name="lenientFutureDates" value="true"/>
       <property name="serverLanguageCode" value="fr"/>
   </bean>

And then let Camel lookup this bean when you use the # notation in the url.

from("ftp://foo@myserver?password=secret&ftpClientConfig=#myConfig").to("bean:foo");

[Important]Important

See File2 as all the options there also applies for this component.

Here are some examples of FTP endpoint URIs:

ftp://someone@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true
ftp://someoneelse@someotherftpserver.co.uk:12049/reports/2008/password=secret&binary=false
ftp://publicftpserver.com/download
[Warning]FTP Consumer does not support concurrency

The FTP consumer (with the same endpoint) does not support concurrency (the backing FTP client is not thread safe). You can use multiple FTP consumers to poll from different endpoints. It is only a single endpoint that does not support concurrent consumers.

The FTP producer does not have this issue, it supports concurrency.

[Tip]More information

This component is an extension of the File2 component. So there are more samples and details on the File2 component page.

The FTP consumer will by default leave the consumed files untouched on the remote FTP server. You have to configure it explicit if you want it to delete the files or move them to another location. For example you can use delete=true to delete the files, or use move=.done to move the files into a hidden done sub directory.

The regular File consumer is different as it will by default move files to a .camel sub directory. The reason Camel does not do this by default for the FTP consumer is that it may lack permissions by default to be able to move or delete files.

The following message headers can be used to affect the behavior of the component

Header Description
CamelFileName Specifies the output file name (relative to the endpoint directory) to be used for the output message when sending to the endpoint. If this is not present and no expression either, then a generated message ID is used as the filename instead.
CamelFileNameProduced The actual absolute filepath (path + name) for the output file that was written. This header is set by Fuse Mediation Router and its purpose is providing end-users the name of the file that was written.
CamelFileBatchIndex Current index out of total number of files being consumed in this batch.
CamelFileBatchSize Total number of files being consumed in this batch.
CamelFileHost The remote hostname.
CamelFileLocalWorkPath Path to the local work file, if local work directory is used.

Fuse Mediation Router supports consuming from remote FTP servers and downloading the files directly into a local work directory. This avoids reading the entire remote file content into memory as it is streamed directly into the local file using FileOutputStream.

Fuse Mediation Router will store to a local file with the same name as the remote file, though with .inprogress as extension while the file is being downloaded. Afterwards, the file is renamed to remove the .inprogress suffix. And finally, when the Exchange is complete the local file is deleted.

So if you want to download files from a remote FTP server and store it as files then you need to route to a file endpoint such as:

from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
[Tip]Optimization by renaming work file

The route above is ultra efficient as it avoids reading the entire file content into memory. It will download the remote file directly to a local file stream. The java.io.File handle is then used as the Exchange body. The file producer leverages this fact and can work directly on the work file java.io.File handle and perform a java.io.File.rename to the target filename. As Fuse Mediation Router knows it's a local work file, it can optimize and use a rename instead of a file copy, as the work file is meant to be deleted anyway.

Camel FTP can operate in two modes in terms of traversing directories when consuming files (for example, downloading) or producing files (for example, uploading):

  • stepwise

  • not stepwise

You may want to pick either one depending on your situation and security issues. Some Camel end users can only download files if they use stepwise, while others can only download if they do not. At least you have the choice to pick.

Note that stepwise changing of directory will in most cases only work when the user is confined to it's home directory and when the home directory is reported as /.

The difference between the two of them is best illustrated with an example. Suppose we have the following directory structure on the remote FTP server we need to traverse and download files:

      /
      /one
      /one/two
      /one/two/sub-a
      /one/two/sub-b

And that we have a file in each of sub-a (a.txt) and sub-b (b.txt) folder.

The following log shows the conversation between the FTP endpoint and the remote FTP server when the FTP endpoint is operating in stepwise mode:

      TYPE A
      200 Type set to A
      PWD
      257 "/" is current directory.
      CWD one
      250 CWD successful. "/one" is current directory.
      CWD two
      250 CWD successful. "/one/two" is current directory.
      SYST
      215 UNIX emulated by FileZilla
      PORT 127,0,0,1,17,94
      200 Port command successful
      LIST
      150 Opening data channel for directory list.
      226 Transfer OK
      CWD sub-a
      250 CWD successful. "/one/two/sub-a" is current directory.
      PORT 127,0,0,1,17,95
      200 Port command successful
      LIST
      150 Opening data channel for directory list.
      226 Transfer OK
      CDUP
      200 CDUP successful. "/one/two" is current directory.
      CWD sub-b
      250 CWD successful. "/one/two/sub-b" is current directory.
      PORT 127,0,0,1,17,96
      200 Port command successful
      LIST
      150 Opening data channel for directory list.
      226 Transfer OK
      CDUP
      200 CDUP successful. "/one/two" is current directory.
      CWD /
      250 CWD successful. "/" is current directory.
      PWD
      257 "/" is current directory.
      CWD one
      250 CWD successful. "/one" is current directory.
      CWD two
      250 CWD successful. "/one/two" is current directory.
      PORT 127,0,0,1,17,97
      200 Port command successful
      RETR foo.txt
      150 Opening data channel for file transfer.
      226 Transfer OK
      CWD /
      250 CWD successful. "/" is current directory.
      PWD
      257 "/" is current directory.
      CWD one
      250 CWD successful. "/one" is current directory.
      CWD two
      250 CWD successful. "/one/two" is current directory.
      CWD sub-a
      250 CWD successful. "/one/two/sub-a" is current directory.
      PORT 127,0,0,1,17,98
      200 Port command successful
      RETR a.txt
      150 Opening data channel for file transfer.
      226 Transfer OK
      CWD /
      250 CWD successful. "/" is current directory.
      PWD
      257 "/" is current directory.
      CWD one
      250 CWD successful. "/one" is current directory.
      CWD two
      250 CWD successful. "/one/two" is current directory.
      CWD sub-b
      250 CWD successful. "/one/two/sub-b" is current directory.
      PORT 127,0,0,1,17,99
      200 Port command successful
      RETR b.txt
      150 Opening data channel for file transfer.
      226 Transfer OK
      CWD /
      250 CWD successful. "/" is current directory.
      QUIT
      221 Goodbye
      disconnected.                                                 
      
    

As you can see when stepwise is enabled, it will traverse the directory structure using CD xxx.

The ANT path matcher is a filter that is shipped out-of-the-box in the camel-spring jar. So you need to depend on camel-spring if you are using Maven. The reason is that we leverage Spring's AntPathMatcher to do the actual matching.

The file paths are matched with the following rules:

  • ? matches one character

  • * matches zero or more characters

  • ** matches zero or more directories in a path

The sample below demonstrates how to use it:

<camelContext xmlns="http://camel.apache.org/schema/spring">
    <template id="camelTemplate"/>

    <!-- use myFilter as filter to allow setting ANT paths for which files to scan for -->
    <endpoint id="myFTPEndpoint" uri="ftp://admin@localhost:20123/antpath?password=admin&ecursive=true&elay=10000&nitialDelay=2000&ilter=#myAntFilter"/>

    <route>
        <from ref="myFTPEndpoint"/>
        <to uri="mock:result"/>
    </route>
</camelContext>

<!-- we use the AntPathMatcherRemoteFileFilter to use ant paths for includes and exlucde -->
<bean id="myAntFilter" class="org.apache.camel.component.file.AntPathMatcherGenericFileFilter">
    <!-- include and file in the subfolder that has day in the name -->
    <property name="includes" value="**/subfolder/**/*day*"/>
    <!-- exclude all files with bad in name or .xml files. Use comma to seperate multiple excludes -->
    <property name="excludes" value="**/*bad*,**/*.xml"/>
</bean>
Comments powered by Disqus