Performance

The MTP/IP networking technology that transports data for ExpeDat is designed to adapt to a wide variety of network circumstances.  The default settings will allow you to use the full available resources of your network while still being fair to other traffic.

If you are not getting the performance you expect, read this section carefully or contact technical support.

In addition to the information contained in this manual, detailed technical information about optimizing network performance can be found in the DEI Tech Notes.  An index of the Tech Notes most relevant to performance can be found at the bottom of this page.

The sections below summarize the network and data storage factors most likely to affect ExpeDat, and describe techniques for increasing or decreasing ExpeDat's speed.

Network Limitations

Most Wide Area Network paths involve hundreds of different components, any one of which may be limiting your available bandwidth.  Here are a few of the more common types to consider:

ExpeDat cannot move data faster than the slowest of these capacities.  Rate-limiting devices, especially firewalls, emulators, and routers, may not be treating all traffic equally!  So if performance is not meeting your expectations, check your systems for the types of limitations listed above.  DEI Tech Note 0003 offers detailed advice on testing and finding network limitations.

Storage Limitations

Filesystem bottlenecks can severely limit performance.  This is particularly true for network attached storage.  Below are some tips for optimizing storage performance:

Any time the total network throughput may be faster than about 100 megabits per second or you are using any form of network attached storage, you should investigate your filesystem's I/O capabilities.  See Tech Note 0023 for further hardware recommendations.

CPU Limitations

Each real CPU core (or 2 vCPUs) can support about one gigabit per second of throughput with content encryption enabled.  A 10 gigabit per second data transfer with encryption requires about 10 real CPU cores or 20 virtual CPU cores.  This does not include the needs of other software which may be running on the same system.

If other CPU intensive applications are running on the same system, network performance may be reduced.  This may happen even if there are idle CPU cores available.  Applications which filter or monitor network traffic may substantially reduce network performance.  For example, running Windows Performance Monitor may reduce network throughput by 25% or more.  For the best performance, avoid running anti-virus, firewall, or monitoring software on the same hardware as ExpeDat.

Operating systems may limit CPU performance under some conditions, such when the device is operating on battery power or when the user is interacting with another application.  Many Linux distributions configure CPUs to operate in power conservation mode by default.  See Tech Note 0035 for instructions to check and adjust the Linux scaling governor.

Security Software

Some anti-virus software, including Windows Defender and all network monitoring software, can greatly reduce data transfer performance.  Symptoms include poor network speeds accompanied by high CPU and filesystem usage, long delays near the end of a download, and slow scan rates when sending folders with large numbers of files.  If you are experiencing slow performance, particularly on a Windows system, test whether anti-virus or other security software is the cause by temporarily disabling it to see if performance improves.

The best way to avoid problems with security software is to offload its functions to a purpose-built device such as a firewall or hardware VPN with sufficient capacity to handle the throughput you require.  If security software must share the same hardware as ExpeDat, white-list the ExpeDat applications and file transfer directories.  If applicable, configure anti-virus software to delay scanning new files until after they are finished writing, disable heuristic analysis, and disable or make an exception for scanning network packets.

Moving Data Even Faster

Normally MTP/IP makes an effort to allow other traffic on the network a fair share of available resources.  If you want your ExpeDat data flows to achieve the absolute maximum speed, regardless of other users, you may do so by increasing the network "aggression" of MTP/IP.

Aggression affects relative performance compared to other data flows sharing the same network path, similar to prioritization.  If there are no other data flows, or if performance is limited by hardware or policy, then increasing aggression may have little or no effect.

Aggression is represented by a number between -3 and +5 in user interfaces, or 125 to 133 in SDKs and logs, which determines how hard MTP/IP presses the limits of the underlying network hardware.  The default value is +2 (130) which tries to strike a balance between speed and fairness.  It is very rare to need to set aggression higher than +2.

Elevated aggression may cause reduced performance or loss of connectivity!
Aggression +4 and higher will always create packet loss.  Never set an aggression level of 4 or higher without careful testing.  Always test at level 2 before trying higher levels.

Setting aggression to +3, +4, or +5 may improve ExpeDat throughput, but it may also be disruptive to other users of the network.  Such high aggression levels can sometimes result in worse performance if firewalls or other bandwidth management devices are present.

Carefully read and understand the following paragraphs before using aggression +5 or MinDatagram.

MTP will automatically attempt to utilize Jumbo frames (MTU 9000) whenever a transfer exceeds about 1.1 gigabits per second.  If every device along the path supports Jumbo frames, the MTP datagram payload size will switch to 8192 bytes for as long as that transaction speed remains high.

If you are certain that every device in your network supports Jumbo frames (MTU 9000) or Super Jumbo frames (MTU up to 65535), then you may be able to reduce I/O overhead and further improve speed by manually advising MTP/IP to use larger datagrams.  This can be accomplished by setting MinDatagram to 8192 for Jumbo frame networks or up to 61440 for Super Jumbo.  Setting aggression to level 5 will also attempt to use larger datagrams, but setting MinDatagram is preferred if you are sure the network is capable.  If any component of the path does not fully support large datagrams, their use may cause severe performance degradation or loss of connectivity.

Refer to each applications's documentation for instructions on setting aggression and Tech Note 0005 for more information about MTP/IP bandwidth management features.

Moving Data Slower

For shared network environments, you may wish to limit ExpeDat's impact on other users even more than the default settings.

The most flexible way to give priority to other users is to reduce the aggression level described above.  Values below 2 will limit the resources MTP/IP is willing to use, leaving more resources for other traffic flows.  Note that on networks with inherently high-latency or packet loss (such as satellite or packet radio), lower aggression may reduce performance even when there is no other traffic.

Aggression affects relative performance compared to other data flows sharing the same network path.  If there are no other data flows and the network path is "clean", decreasing aggression may not have an obvious effect on performance.  But it will still cause performance to slow down in the presence of other data flows.

You can also set specific bandwidth limits for ExpeDat.  The MaxRate settings can be used to establish speed limits for both individual transactions and the server as a whole.  If your data path has a fixed known capacity, you can set MaxRate to a value less than that to apportion the bandwidth between ExpeDat and other users.  You can even combine this with increased aggression to throttle ExpeDat to exactly that amount of bandwidth.

If you have applications on your network which are sensitive to latency issues, such as voice or video over IP, you can set a specific latency ceiling for ExpeDat.  The MaxRTT setting will cause MTP/IP to slow down whenever it observes latency above the given value.

In order to avoid stalls and other inefficiencies, MTP will always maintain at least one datagram on the network at any time.  Thus, transactions cannot be throttled below 256 bytes per Round-Trip-Time.  For example, a path with an RTT of 1ms has a minimum rate limit of about 2 megabits per second, while a path with 10ms latency has a minimum of about 200 kilobits per second.  This affects both MaxRate and MaxRTT, primarily on LANs.

Refer to the documentation for servedat, ExpeDat Desktop, and movedat for details on how to set these values.  Also see DEI Tech Note 0005 for more information about MTP/IP bandwidth management features.

Client, Server, User, and Network Settings

Performance can be regulated at either the client or server.  Server-side settings can be targeted to specific users or client networks.  Most server-side performance settings have a default value and a limit value.  The default is used when the client does not specify a value, but the client may not go past the limit value.

Many server performance settings can be set according by the source IP address of the client using the SiteOptions configuration variable.  This allows administrators to regulate performance based on the client network.

Many server performance settings can be set according to the authenticated username by specifying the appropriate options in the AuthFile or AuthHandler.

The server MaxRate settings are applied to individual network transactions.  The MaxRateTotal settings are enforced against all transactions in aggregate.

The ExpeDat Desktop client also allows administrators embed settings within a copy of the application.  This allows different performance profiles to be distributed to specific persons or systems.  Each embedded settings can be locked to prevent the user from modifying it.

See Also

The following Tech Notes provide further guidance on performance tuning:

0023 Recommended minimum hardware for high speed networking.
0005 Features for regulating MTP and ExpeDat bandwidth usage.
0029 Optimizing Network Attached Storage (NAS).
0003 Important tips on conducting accurate and meaningful performance tests.
0033 Instructions for capturing network diagnostics.
0021 Guidance for accurately measuring network Loss, Latency, and Speed.
0022 See this note if you are considering use of a network emulator for testing.
0009 Details about specific devices and factors which may degrade network performance.
0024 Operating system tuning for certain unix variants.
0018 Configuration requirements for RAIDs.
0014 Pros and cons of using inline compression.
0025 Using ExpeDat with Amazon Web Services EC2 instances.
0035 Linux Performance Tuning.
0036 MTP Performance Statistics.