MTP/IP is designed for integration into the widest possible variety of system and environments.  This flexibility is supported by multiple applications and interfaces which can be used alone or in combination.  The following guidelines may assist developers in choosing which approaches are most suitable for integrating MTP/IP network acceleration technology into any given workflow.

Embedded Executables

The simplest way to integrate MTP/IP technology into an existing workflow is to embed an off-the-shelf DEI product alongside the target application.  All DEI products are available in the form of small, single executables which can be called from scripts, batch files, or other executables.

ExpeDat, in particular, has many features specifically designed to enable tight integration through its command-line interfaces.  These features include detailed exit codes, machine parseable logs and progress updates, standard I/O streaming, and more.  The ExpeDat server can also call on third-party applications and scripts to process data or perform automation.  See Tech Note 0015 for details about Embedding ExpeDat.

Primary Target

Existing file or object based workflows, particularly those which already interface with external utilities like FTP or curl for data transport.

Advantages

Rapid plug-in deployment, sometimes in as little as 5 minutes.  Little or no modification to legacy source-code.

Disadvantages

Only suitable for transferring files, objects, or standard I/O data streams.  Requires installation of ExpeDat client and/or server executables.

ExpeDat Client SDK

The ExpeDat Client Software Development Kit (EXP) is a C language library which can be used to build custom ExpeDat client software.  DEI's own ExpeDat and SyncDat clients are built using EXP, giving your application access to the same functionality as our own.

EXP provides simple, high-level interfaces for communicating with an ExpeDat server.  This includes sending and receiving files, authentication, encryption, directory listings, file maintenance, and even streaming file content to or from memory.  Once the parent application starts a file transfer, it does not need to perform any further actions until it is notified that the transaction has completed.

Primary Target

New or existing applications whose primary function is file transfer.  This includes not only traditional FTP style clients, but also managed file transfer systems (MFTs), content delivery networks (CDNs), and any application which may move files along a client-server model.

Advantages

Source code level integration allows the highest level of performance and the most user interface control and flexibility.  Data can be delivered to or from any source, including disk, memory, or streams.

Disadvantages

Requires installation of an ExpeDat server component at the server side.  Focused on client-server file transfer and so not suitable for general data exchange between peers.

MTP Tunnel

The MTP Tunnel port forwarding utility captures connections from TCP clients to TCP servers, transporting the data across the network using MTP/IP as an intermediary.  This allows you to accelerate off-the-shelf and legacy TCP applications without changing the existing software or workflows.  Any client-server TCP session involving large amounts of continuous data flow can be captured by directing it to an outgoing tunnel port, similar to Secure Shell port-forwarding.  Client applications can also be forwarded via stdio for embedded solutions.

Primary Target

Legacy and off-the-shelf TCP applications performing high-volume data transport.

Advantages

Works with existing TCP clients and servers, regardless of purpose or data type.

Disadvantages

Only suitable for high-volume, continuous data flows.  Applications which produce data slowly or are "chatty" may not be accelerated.  Requires significant RAM when used on high-speed data paths.

Document Exchange SDK

The Document Exchange Software Development Kit (DOC) is a C language library which provides session management and data delivery for general purpose data transfer between networked applications.  DEI's own end-user products are built on top of DOC, providing a consistent platform for inter-application communication.

The DOC SDK is focused on the idea of sending or retrieving blocks of data which may be located on disk, in memory, or in sequential streams (see Sequencing below).  It provides a high-level model for both client and server networking, allowing the parent application to focus on data management.  Once an application initiates or accepts a document transfer, it does not need to perform any further actions until it is notified that the transfer has completed.

Primary Target

General data transfer between applications, including client-server, peer-to-peer, and one-to-many.

Advantages

Complete integration and control over all components, regardless of communication model.  "Fire-and-forget" data transfer minimizes the need for network level design awareness.  Data can be delivered to or from any source, including disk, memory, or streams.

Disadvantages

Developer is responsible for providing all data management functionality, such as authentication, data storage, user interfacing, encryption, and so on.

Sequencing SDK

The MTP Sequencing Software Development Kit (SEQ) is a C language library which allows data being delivered via MTP/IP to be buffered and accessed using a read/write data model.  This allows program generated data to be sent, or incoming data to be processed sequentially and entirely within memory.  DEI's own end-user products use SEQ for tasks such as compression and decompression, listing directory trees, and streaming standard I/O.

The SEQ SDK can be used as a data target for any of the other DEI Software Development Kits, including the ExpeDat Client SDK, Document Exchange, and Core MTP/IP.  It can also be used for general purpose intra-process buffering, independent of the network.  In addition to basic read/write functionality, it also provides asynchronous notification, random buffer access, flow-control, and the ability to examine buffer capacities.  If you are used to programming against TCP/IP sockets, the SEQ interface will be at once familiar and liberating.

Primary Targets

Supplement for streaming data in applications using other MTP SDKs.  Legacy interface for existing applications written against TCP/IP.

Advantages

Traditional read/write access.  Many sophisticated features that are lacking in typical sockets style interfaces: asynchronous notification, full buffer access, peek/poke, and more.

Disadvantages

Each SEQ construct requires allocation of a memory buffer that is typically several megabytes in size.  Networks which are very fast and/or very high-latency may suffer performance limitations if the buffer is inadequate.

Core MTP/IP

The Core MTP/IP Software Development Kit (Core MTP) is a C library providing access to the most basic building blocks of DEI technology.  MTP/IP transactions are highly optimized to deliver maximum network performance with minimum overhead.  For example, an entire network transaction can be reliably completed in a single round-trip time with no setup or tear-down overhead.

Implementing on top of the Core SDK delivers maximum efficiency in both network and CPU utilization, and maximum flexibility to achieve that efficiency for almost any network communication task.  However, with that flexibility comes a great deal of design responsibility.  Developers working directly with Core MTP/IP need to have a strong understanding of how packet-switched IP networks operate and experience designing network applications.  MTP/IP will handle the details of flow-control and error-recovery, but it is up to the developer to decide issues such as session management, authentication, data processing, and so on.

Primary Target

Applications requiring rapid transaction processing, precise network control, and the tightest possible source code integration.

Advantages

Complete freedom to create a highly optimized network application.

Disadvantages

Requires an advanced understanding of network transaction processing and software design.

Choosing

Most MTP/IP integrators find it simplest to begin with Embedded ExpeDat.  This provides the fastest integration path and, for most file transfer applications, it provides more than adequate functionality.  For projects which require maximum client-side integration, the ExpeDat Client SDK provides source-code level control without requiring the programmer to delve into network level details.

Developers seeking to create or accelerate general purpose network communication paths will find that the Document Exchange and Sequencing SDKs provide that functionality with minimal network level consideration.  The Core MTP/IP SDK is for advanced network programmers who wish to create their own highly-optimized communication software.

Integrators seeking to accelerate legacy applications, where source code is not available and ExpeDat's file-transfer model does not fit, may consider MTP Tunnel as a way to capture and accelerate existing TCP/IP sessions.

For more information about MTP/IP integration options, including a consultation with one of our engineers and a free trial of any SDKs, contact us.