Market
File transfer market
Despite opinions from various IT experts announcing for years the imminent collapse of the file transfer software market in directions of IT services, we observe that the number of transfers is steadily increasing. At the same time, new functionality needs are emerging and become essential. The new requests that we receive more and more often concern possibilities for sending not only files but also serializable objects stored in the Cloud or in local FS, thus exchanging type InputStream -> OutputStream, taking into account Unicode data code conversions, etc. … This means that the file transfer market (and not only) is still there, evolving, and growing and software publishers of this type of products must remain vigilant to be able to respond to new needs while maintaining compatibility with solutions used by many exchange partners.
Protocols
In the current landscape of file transfer standards and protocols, two major groups of protocols can be distinguished based on their origins and purposes. Obviously, all these standards have the ability to transfer files, but for some, this functionality is the end goal in itself, while for others, transferring a file is not enough. Awareness of additional needs is not innate. It belongs to the heads of IT services of companies where the management of very strong transfer flows clashes with the needs of their IT production services or those who are very concerned with ensuring the security of their flows beyond securing transport through the use of TLS or SSH tunnels.
“Classic” file transfer protocols like FTP(S), SFTP are already old. The community origin of these protocols meant that their purposes were limited to their primary purpose – file transfer. Evolutions brought by the community (such as the addition of the SITE command in FTP) open up new possibilities but are not enough to change the nature of these protocols. Their purpose remains the same. On the other hand, proposals for the evolution of these protocols are continuously being discussed, but it cannot be the same with their implementation in software so that the innovations can be put into practice.
Awareness of much more complex needs was the basis for defining a much more advanced protocol such as the PeSIT protocol (specifications available at www.pesit.org). A working group of file transfer experts, IT production, and telecom specialists from a large grouping of financial institutions focused on the subject and developed a protocol that allows not only transferring files between different systems but also organizing the file transfer service by taking into account the identification and classification of each transfer, managing transfers in “store and forward” mode with tracking end-to-end, performing application acknowledgment of file reception, managing transfer priorities, and organizing flow control beyond transport-level flow control. Exchange of messages and user fields, and classification of flows by application identification help anchor each transfer better in business applications exploring exchanged data. These needs are evident for file transfer service managers in financial, insurance, or large distribution institutions and escape the consciousness of IT services managers having made some hundreds of transfers per day. They have never had to ensure the recovery of tens, cents of thousands of files waiting on their own server while facing the incessant flow of external transfer requests, which often happens after a network incident. There are services that must have the ability to transfer millions of files per day and do so under strong constraints of their exploitation time by the production service.
Another very interesting protocol was also designed for the needs of a financial institution to ensure flawless security of bank client transactions. The file transfer is one of multiple features of the EBICS protocol (specifications available at www.ebics.org) so it cannot even be said that it is a file transfer protocol. It is a protocol that is carried by HTTPS flows where each HTTPS request/response constitutes one EBICS transaction authenticated, encrypted and signed by appropriate certificates. These certificates are exchanged only once during the EBICS connection phase which takes place when opening the account or when renewing/changing the certificate. The fact that public keys do not circulate on the network is an undeniable advantage from a security point of view. For its ease of implementation (it is always the HTTPS flow that needs to be passed through all network DMZs), it is used to ensure the transfer of files where data security and strong authentication of protagonists are vital.
Our approach
Our rich experience in the field of file transfer, both on the architecture and product development side, as well as on the organization of file transfer services and crisis management in very intense production, has made us understand the impact of the protocols and infrastructure choices on the productivity of large-scale file transfer services. These choices, which must be made at the central entity level of exchanges, often impose the use of the same protocol by clients or peripheral entities. They can be dictated by service management efficiency and capabilities, as well as by the purchase and maintenance costs of the central solution and the availability and prices of peripheral solutions. On the central side, low-cost solutions based on the use of FTP or SFTP for mass exchanges are accepted despite productivity declines because peripheral users can be equipped with widely available freeware solutions.
This is not yet the case for the PeSIT protocol, which is by far the most appropriate protocol for organizing real mass file transfer services. Our goal is to provide public protocol connectors starting with PeSIT to make its implementation easier for publishers, integrators, major accounts and other users who hesitate to implement it. PeSIT Client Léger, which is still a paid solution, is an independent solution aimed at the needs of the so-called peripheral population.
Our partner Evoleen.byGecko’s MFT X-Protocols solution already integrates our PeSIT connectors to take the best from our software. All of these offers aim to promote the PeSIT protocol, which we strongly believe deserves to be better known than it is today. As part of this promotion, we have published the latest protocol specification on the www.pesit.org site to keep it as a public protocol as it always has been. We also plan to open a new site www.pesit.net which will allow users of this protocol, all market products combined, to submit proposals for protocol evolution, follow their acceptance and their implementation in market products.
We have the same approach to the EBICS protocol except that in this case, the protocol is well-known and maintained by appropriate organizations.
To complete our offer, we will propose connectors with similar interfaces for other public protocols such as FTP(S), SFTP, HTTP, etc. …
Do not hesitate to contact us for more information. We remain at your service