Sneakernet

A USB flash drive is physically much smaller than a floppy disk, but it has a capacity thousands of times larger.

Memory cards are a popular physical medium for transferring files, and have become smaller in size as technology has advanced

Sneakernet is an informal term describing the transfer of electronic information, especially computer files, by physically moving removable media such as magnetic tape, floppy disks, compact discs, USB flash drives (thumb drives, USB stick), or external hard drives from one computer to another. This is usually in lieu of transferring the information over a computer network. The name is a tongue-in-cheek sound-alike to Ethernet, and refers to the use of someone wearing sneakers as the transport mechanism for the data.

Sneakernet, whether called that or not, is often used as an academic example to illustrate long ping times, and the trade-off between latency and bandwidth.

Summary and background

Sneakernets are in use throughout the computer world. Sneakernets may be used when computer networks are prohibitively expensive for the owner to maintain, in high-security environments where manual inspection (for re-classification of information) is necessary, where information needs to be shared between networks with different levels of security clearance, when data transfer is impractical due to bandwidth limitations, when a guest laptop is incompatible with the local network, or simply when two computers are not powered up at the same time or lack the correct interconnecting cabling. Because sneakernets take advantage of physical media, different security measures must be taken into account for the transfer of sensitive information.

This form of data transfer is also used for peer-to-peer (or friend-to-friend) file sharing and has grown in popularity in metropolitan areas and college communities, sometimes for the purpose of distributing works of authorship. The ease of this system has been facilitated by the availability of USB external hard drives, USB flash drives and portable music players.

The United States Postal Service also offers a Media Mail service for compact discs among other items. This provides a viable mode of transport for long distance sneakernet use. In fact, when mailing a sufficiently large hard drive or a spindle of DVDs, the throughput (amount of data per unit time) may compete favorably with other methods of data transfer.

Theory

From an information theory standpoint, sneakernets can achieve tremendous throughput, but they suffer from high latency (see Comparison of latency and throughput). The throughput of the network is directly proportional to the size of the transmitted file(s). Latency is based on the amount of time it takes to fully process the request for information. Latency would include the time it takes to write the storage media and the time to travel from point A to point B.

For example: Person A requested Person B to send them a DVD (4.7 GB) worth of information. Over the Internet, the latency for the file request may be milliseconds, but at a modest broadband download speed of 50 kB/s it may take up to a day to complete the transfer. On the other hand, Person B could burn a DVD and deliver it to Person A in an hour. The latency was an hour, but the throughput of the transfer is roughly equal to a transfer rate of 1305 kB/s.

The theoretical capacity of a Boeing 747 filled with Blu-ray Discs is 595,520,000 Gigabytes, resulting in a 245,829 Gbit/s flight from New York to Los Angeles.

Similarly, as of 2010 the highest capacity backup tape format available is LTO-5, with a capacity of 1500 GB. If a tape of this capacity were sent by overnight mail and were to arrive around 20 hours after it was sent, the effective data rate would be 166 Mbit/s. With networking technology, this magnitude of speed over this distance would be very difficult to attain without a costly dedicated connection as one would likely need to use several hops and have a connection that is not oversubscribed.

Sneakernets may also be used in tandem with computer network data transfer to increase data security. For example, a file or collection of files may be encrypted and sent over the Internet while the encryption key is printed and hand delivered or mailed. This method greatly reduces the possibility of an individual intercepting both the key and encrypted data.

Another way sneakernets are used together with network transfers is to provide an initial full backup as a base for incremental backups. In the case of a large (several terabyte) dataset that grows by just a few megabytes a day, the initial seeding of the data to be backed up would require an excessively long time upload over a network. One solution is to make a local copy of the data which is then physically relocated to the remote location and topped up with daily incremental backups over a network.

Usage examples

• The May 2011 raid of Osama Bin Laden's compound revealed that he used a series of USB thumb drives to store his email drafts. A courier of his would then take the saved emails to a nearby Internet cafe and send them out to the desired recipients.
• In September 2009, Durban company Unlimited IT reportedly pitted a carrier pigeon against South African ISP Telkom to transfer 4 GB of data 60 miles (97 km) from Howick to Durban. The pigeon, carrying the data on a memory stick, arrived in one hour eight minutes, with the data taking another hour to transfer off of the memory stick. During the same two-hour period, only about 4.2% of the data had been transferred over the ADSL link. A similar experiment was conducted in England in September 2010; the "pigeonnet" also proved superior.
• North Korea's national administration has used flash drives to send confidential data to the United Nations.
• Google has used a sneakernet to transport datasets too large for current computer networks, such as the 120 TB of data from the Hubble Space Telescope.
• The SETI@home project uses a sneakernet to overcome bandwidth limitations: data recorded by the radio telescope in Arecibo, Puerto Rico is stored on magnetic tapes which are then shipped to Berkeley, California for processing. In 2005, Jim Gray reported sending hard drives and even "metal boxes with processors" to transport large amounts of data by postal mail.
• Many film editing and visual effects companies transfer large film-scans using hard drives shipped via courier (to reduce bandwidth bills, and to reserve bandwidth for more time-critical transfers).
• Wizzy Digital Courier provides Internet access to schools with poor or no network connectivity by implementing UUCP on USB memory sticks. This allows offline cached email transport and scoops of web pages that back-fill a web cache.
• When home broadband access was less common, many people downloaded large files over their workplace networks and took them home by sneakernet. Today when home broadband is more common, sometimes technical workers at institutions with congested WAN links do the reverse: downloading data at home in the evening and carrying the files to work on USB flash drives.
• In the early demoscene, the primary method of exchanging data was using snail mail to exchange floppies between groups. Each group usually had at least one person designated as a swapper, who would exchange news, data and productions with swappers from other groups this way. The best swappers were known to send and receive over 100 mails a month.
• Online DVD rental services such as Netflix and GameFly are effectively sneakernets, as they deliver data on DVDs and occasionally other forms of digital media via regular postal mail.
• Petroleum seismic surveys routinely record field data which are many terabytes in size. A cluster computer is required to process these data, and may take a year or more, during which time the field crew will wish to work in other areas. The field data are generally hand-carried on tape, and increasingly on hard disk inserts, to the processing centre.
• Data analytics teams in the financial services sector often use sneakernets to transfer sensitive corporate information and information obtained from data mining, such as ledger entries, customer data, and financial statistics. There are several reasons for this: firstly, sneakernets can generally provide very high security (and possibly more importantly, they are perceived to be secure) due to the impossibility of a MITM Attack or packet sniffing, even if the receiving or sending computational device is compromised via malware; secondly, the volumes of data concerned are often extremely high; and thirdly, setting up secure network links between the client business and the analytics team's facilities is often either impossible or an extremely convoluted process.
• Very Long Baseline Interferometry performed using the Very Long Baseline Array ships hard drives to a data reduction site in Socorro, New Mexico. They refer to their data transfer mechanism as "HDOA" (Hard Drives On Airplane).
• The Rigsum Sherig Collection project uses a sneakernet to distribute offline educational resources, including Kiwix and Khan Academy on a Stick, to hundreds of schools and other educational institutional in the Kingdom of Bhutan. Many of the schools in Bhutan have computers or IT labs, but no Internet connection (or a very slow one). The sneakernet, facilitated by teachers, distributes about 25 GB of free, open-source educational software to the schools, often using external hard disks.

In media

Non-fiction

Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

—Tanenbaum, Andrew S. (1989). Computer Networks. New Jersey: Prentice-Hall. p. 57. ISBN 0-13-166836-6.

The original version of this quotation came much earlier; the very first problem in Tanenbaum's 1981 textbook Computer Networks asks the student to calculate the throughput of a St. Bernard carrying floppy disks (in 1981 the standard 8 inch floppy held 241 kilobytes of data per side (but various formatting and data blocking techniques could get that as high as 800 kilobytes per side), the 5.25 inch floppy held 180 kilobytes per side). The first USENET citation is July 16, 1985, and it was widely considered an old joke already, possibly dating from the 1970s. Other alleged speakers included Tom Reidel, Warren Jackson, or Bob Sutterfield. The station wagon transporting magnetic tapes is the canonical version , but variants using trucks or Boeing 747s or C-5s and later storage technologies such as CD-ROMs, Blue-Ray DVDs or SD Cards have frequently appeared.

The 1970s "old joke" referred to above was told to this poster in the NASA / Jet Propulstion Lab cafeteria in about 1975-1976. He worked in the Digital Maintenance group in the JPL Space Flight Operations Center from 1974 to 1978. The story / joke was a classic regularly used at JPL to explain ping time, and differentiate bandwidth from latency (and, by the way, the need to document where your cables ran, and that you needed to distribute your data circuits across multiple cables in different trenches - or somehow via multiple paths).

The NASA Deep Space Network tracking station at Goldstone is just outside of Fort Irwin, just east of Barstow, California. When you leave the highway you have to go through Fort Irwin to get to any of the Goldstone facilities. Depending on the highway route taken, and which Deep Space Network dish at Goldstone you are driving to (or starting from) it was about 160-185 miles (255-298 km) from JPL. At freeway speeds (65 mph, about 100 km/h) it was a minimum of three-and-a-half hours, usually four, and frequently more, depending on the traffic. If you ignored the speed limit while out in the desert (risky) you could get closer to three and-a-half hours. This distance and speed also explained how the "ping time" was 7 to 8 hours. Several of the freeways now in existence were not there then.

At the time (early 1970s), the data links from JPL to Goldstone ranged from as low as 1200 and 2400 bps (several of each) to 9600 bps (one or two). The 9-track magnetic tapes of the day recorded at a maximum density of 6250 bits per inch (but some older drives were limited to 800 or 1600 bits per inch). The tape reels were made in different sizes, the largest held about 2400 feet of tape, but due to the data being written in records, with gaps between the records, the maximum data capacity of a 2400 foot reel, blocked at 32,767 bytes per record and recorded at 6250 BPI was 170 megabytes per reel.

As the story that your contributor heard went, one day a plumbing contractor's backhoe dug up and broke the underground cable that carried ALL of the JPL-to-Goldstone data and voice lines through Fort Irwin, and it would take at least a day, maybe longer, to repair. So someone was designated to drive two boxes of 12 reels each of magnetic tape down to JPL, and quickly. The first available vehicle was a white NASA station wagon. Hence the punch line: "Never underestimate the bandwidth of a station wagon full of magnetic tapes hurtling down the highway".

Rounding off the numbers, twenty-four reels of tape at 170 megabytes each is 4080 megabytes. Three and a half hours is 210 minutes. 4080 megabytes divided by 210 works out to about 19.4 megabytes per minute, or 32.3 kilobytes per second (258.4kilobits per second) - over 100 times faster than a 2400 bps data circuit of the time. Note that the incident above involved only 24 reels - which didn't come anywhere near filling the station wagon, in fact the two boxes of tapes didn't even fill the front passenger seat. (as an aside, a station wagon is known as an estate car or estate in other parts of the world).

Incidentally, that conversation was the first time your contributor ever heard the term backhoe fade used to describe accidental massive damage to an underground cable (compare it to the term rain fade used to describe a fade-out of a point-to-point microwave radio path due to the absorptive effect of water in the air).

Fiction

The Terry Pratchett novel Going Postal includes a contest between a horse-drawn mail coach and the "Grand Trunk Clacks" (a semaphore line) to see which is faster to transmit the contents of a book to a remote destination.

The "valuable data file" has become a common MacGuffin in action films and television programs, and indeed the motif of the "valuable letter or documents" (pre-electronic information storage technology) dates back hundreds of years.

• The film Live Free or Die Hard depicts a digital thief attempting to download 500 TB of financial data to (a) suitcase sized package(s).

• The 1995 film Johnny Mnemonic, based on the short story by William Gibson, stars Keanu Reeves as a digital courier with 320 GB of corporate data transported in his head. Gibson's 2007 novel Spook Country also featured sneakernets, with iPods being the storage device used to clandestinely move information.

• In the episode "Amen" of The Newsroom (U.S. TV series), associate producer Maggie Jordan, after being told "it's 2011, we don't run film manually", is told that the video is taking too long to render on the local machine. It cannot feasibly be transferred over the network in raw form, and must be run to the newsroom manually on a USB drive.

• In the James Bond film Skyfall, MI6 sets up a sneakernet due to security concerns.

Similar concepts

• Delay-tolerant networks, such as the Haggle project at Cambridge University
• IP over Avian Carriers (RFC 1149), an April Fools' Day RFC describing the transmission of messages via homing pigeon
• Drew Houston, the inventor of the Dropbox cloud storage and file synchronization system once referred to his invention as "sneakernet minus the removable media".

See also

• Darknet (file sharing)
• Data Mule
• Jargon File
• Meatspace (tongue-in-cheek implication of the real world as a similar analogue to cyberspace writ large)
• Pod slurping
• USB dead drop
• Sideloading

External links

• "Don't Copy That Floppy" in Internet Archive
• Entry for "sneakernet" in the Jargon File
• "Buzzwords Galore and Bandwidth that May Rival Your Stationwagon"