Data Expedition, Inc.
News
Articles, events, announcements, and blogs
Data Expedition, Inc.
News
Articles, events, announcements, and blogs
Maybe not. Our industry is often described with terms like "acceleration" or "high-performance." It sounds good: everybody wants the things they wait on to go faster. Customers naturally come to us seeking faster data transfer speeds. But when it comes to moving data across a network, there are a lot of moving parts. Software is just one of them.
My first question, "Do you really need what I'm selling?"
No software can move data faster than the underlying hardware. So, if you think your data should be going faster, the first place to look is your telecom provider. If you have a 100 megabit per second (Mbps) uplink, and the system at the other side has at least a 100 Mbps downlink, and you are getting speeds around 90 Mbps, then you don't need acceleration software. Ninety percent utilization is about as good as it gets (accounting for network overhead), so the only way to go faster is to get faster lines. Remember that it doesn't help to upgrade just one end: a data path is only as fast as its slowest link.
If you don't seem to be filling your data pipe, make sure your pipe is as wide as you thought. More than half of the people who reach out to us do not have accurate information about the speed of their network connections. Differences between uplink and downlink speeds, corporate bureaucracy, bonded links, and outdated telecom contracts often cause expectations to exceed reality. Transferring data to nearby servers or using public test sites can give you a quick estimate of what your network is capable of, at least some of the time. Also, keep in mind that you may not be the only user of those links: bandwidth is a shared resource and various devices may demand that you not exceed your fair share.
Even if your network has room to spare, network links are not the only components in your data path. Storage plays a big part as well. Spinning hard disks and NAS protocols can severely constrain speeds, especially on the receiving side. Most people assume RAIDs are faster than single disks, and they can be, but most RAID configuration types are actually slower. Operating system buffers can sometimes hide storage problems by masking the true start and end of a transfer, and some data transfer software (Aspera comes to mind) will blithely declare its job done even while gigabytes of data are still waiting to be written (our own file transfer software doesn't claim to be done until data is actually committed to storage). Performance tests need to compare a variety of storage options and be aware of caching issues.
Even when hardware is underutilized, there are plenty of software bottlenecks that can limit your speeds. Firewalls and anti-virus software may decide that your data flow, in particular, does not deserve to go fast. Tracking down those limitations can be especially frustrating when the limits are based on obscure "heuristic" rules or long forgotten configurations made by someone who hasn't been at your company in years.
Data transport acceleration software fixes a very specific problem: underutilization of hardware due to transport protocol inefficiencies. In other words, TCP (and many so-called "accelerated" solutions) often can't fill the pipe. Data transport optimization is about bringing utilization as close as possible to 100 percent. So, before you ask whether that will make your files transfer faster, ask yourself just where that 100 percent benchmark lies.
Tech Note 0009 examines the many bottlenecks that can limit your data transfer and Tech Note 0023 has recommendations for setting up fast systems.
If bureaucracy, shared lines, or missing configurations are making it hard to find out your hardware capacity, please feel free to download a trial of ExpeDat and follow the instructions in Tech Note 0033. Our software's built-in diagnostics can often narrow down the source of a bottleneck and we're always here for a free consultation.