In my last two posts I described how B2B integration infrastructures at major corporations are choking on the increasing number of large files transmitted crossing their firewalls. Most of the existing approaches to large file transfer are expensive, proprietary and complex. It is interesting to note that corporations have yet to achieve a simple, open, universal file sharing model comparable to what consumers enjoyed with the original Napster in 1999. Yet, business users expect to be able to share large files at work the same way they do in their everyday consumer lives. Recent technology advances compound the problem. Broadband connectivity has become nearly ubiquitous in most developed countries. For example, Verizon FiOS offers an Internet package that allows home users to buy Internet connectivity at speeds up to 50Mbps. Furthermore storage costs continue to decline rapidly. Grocery stores now sell 16GB USB memory sticks in the check out aisles for $20. Despite these advances in networking and storage technologies, the corporate world continues to struggling with large file transfer.
A Napster-like File Transfer Service for Business Users
So how do we solve the large file transfer problem in B2B? I think we need a Napster-like service, but for business users. Unfortunately, for many people, the original Napster service reminds them of college students illegally sharing MP3s with one another in an effort to outwit Hollywood’s recording industry. From my perspective, Napster was the key catalyst for the explosion of digital media content that has revolutionized the home entertainment experience. I also credit Napster as a key pioneer in the large file transfer segment. Napster provided end-users with the ability to exchange high volumes of MP3 files across the Internet.
Napster changed the World
Although each MP3 was only a few megabytes in size, the capability was revolutionary at the time it was introduced. If only we could create a file transfer service that was as easy-to-use, as widely-deployed and as reasonably priced as Napster, but for corporate use. Key features would be:
- Ease of use – Users familiar with Internet browsing would be able to upload or download. There would be no need to write file transfer scripts or otherwise engage IT resources.
- No vendor lock-in – No proprietary communications protocols would be used to perform the file transfers. Open standards would be used. Furthermore no desktop software clients would be necessary.
- Low cost – The service should be considerably cheaper than the cost of licensing MFT software packages. Users should have the option of paying per file transmission or subscribe to unlimited usage for a monthly recurring fee.
The introduction of such a commercial service would create a viral effect that promotes widespread adoption quickly throughout the business community. There are a few legitimate concerns that business users would have with sharing their critical IP and sensitive documents over a Napster-like service. Therefore, I would propose a few key additional services for business users:
- Store-and-Forward – Files would only be housed on the hosted servers temporarily. Once the transmission to the receiving entity is completed only metadata such as file size, sender, receiver, date/time would be retained for tracking purposes.
- Data Security – Users could encrypt sensitive file transmissions to protect sensitive corporate IP. The service should offer higher levels of security such as 128 bit SSL encryption in a premium package.
The good news is that such a B2B file sharing service is not far-fetched. With the recent explosion in cloud computing numerous highly scalable, secure, reliable file transfer services have been introduced by start-ups. Hosted file sharing services such as Gigasize, Leapfile, YouSendIt and Rapidshare are available today with relatively large transmission capabilities and very affordable prices. We have also introduced a first generation file sharing service at GXS. But, perhaps, most interesting is a cloud-computing file transfer service that I will detail in my next post.
5 Responses to “We Need a Napster-like Service for B2B File Transfers”
I think Napster (and its imitators) had a pretty good approach to user interface, that’s for sure.
As a secondary note– just today we were contacted by an interested party wondering why we use UDP instead of Torrent technology. I only glossed over it in my comment above, but the bottom line is that torrents, like Napster, require many peers to function properly.
I can’t name a single commercial enterprise (or any enterprise concerned with reliability, predictability, and security) that would have hundreds of its end users (ie. employees) keeping their machines powered on in order to seed files for sharing throughout the corporation.
Not to mention, you would need to set up a private tracker site.. and on top of that, even though the tracker site might be private, anyone could -in theory- get a hold of the .torrent file, making it a completely insecure way to share files. As we all know, enterprises often have security as their PRIMARY concern, with transfer speeds and ease of use being secondary concerns.
Anyhow, just thought I’d mention it in case some of your readers out there are wondering why a company would choose UDP (one of the protocol layers at the core of the internet) instead of a torrent-based approach.
Don’t get me wrong, I love Torrent. It makes acquiring the ISOs of the latest Linux distributions a snap. But the right tool for the right job, and Torrents are best when used for community-driven file distribution.
Greg
Some interesting comments about the Napster idea. I should have expanded upon this in my post, but there are definitely some aspects of Napster that are not required for B2B file exchange. As Greg points out, the demand from users is really for one-to-one, one-time file transfer. Most Napster file sharing scenarios involved many users downloading the same file. What I did like about the Napster model was the simplicity, low cost and widespread usage.
I wasn’t familiar with File Catalyst, but after reviewing the web site I am impressed with the features. I especially like some of the use case scenarios such as sharing electronic medical records and video game source code. These are the types of collaborative B2B file sharing scenarios that I referred to in earlier posts.
Excellent tip about the ADC protocol Martin. I will research this further as it seems to closely resemble the type of file transfer standard I was proposing.
Finland is setting the example for the rest of us. National Post services are legally required to pick up and deliver from every address. Why shouldn’t telecom carriers have to provide broadband to every household as well.
The most important thing is for the complexity to be as minimal as possible for the end users. Even when the server-side software might require some technical know-how to install, if the end user experience is simple as pie, then the most important goal has been met.
So, I think everyone would agree with that, which means that the next thing to solve is the issue of protocl.
The problem with a Napster-like solution is the lack of peers. Napster worked (works…? what is their current technology?) because you could have, say, 2000 people with the Engelbert Humperdinck song that you want to download, all giving you pieces of it. This doesn’t fit into business very easily because most business-related files aren’t shared between so many peers.
If it could be peer-dependent, Torrenting is the way to go. The technology is already available, but seemingly only used by businesses with something to share to many, such as makers of Linux distributions (great for torrenting).
Since business file transfer ISN’T usually peer-dependent, what you’re left with, no matter how you package the look and feel, is the equivalent of instant-messenger-style file transfer. Two peers connect to each other with a simple client and the file is exchanged.
I’m not sure to what extent it would be polite to post links on your blog, so I’ll err on the side of politeness– suffice it to say, there are companies that provide organizations with the ability to create a “YouSendIt”-like experience on their own infrastructure. All the ease of use, without the “my data is on someone else’s server” security issues.
We hopefully agree that the end-user experience is pretty much goal #1, but if the “shape” of the file transfer is going to change (to make a Napster-like application or something equally simple), the protocol needs to change to accomodate that goal. Torrent or peer-to-peer are essentially highly-multiple-streamed HTTP transfers, and overcome the limitations of HTTP (TCP) by sending hundreds of chunks from different sources. But for B2B file transfers without so many peers, an alternative to this is needed right at the protocol layer. The obvious choice (and the one used by acceleration-enabled companies like FileCatalyst, Aspera, Rocketstream, etc disclosure: yes, I work for one of those companies) is to create UDP-based solutions. Which are proprietary, something that Steve and likely others won’t always be 100% comfortable with.
The best scenario to the end-users? Something that “feels” as simple as some of the hosted solutions, possibly deployed on the cloud, and which enables (admittedly proprietary) UDP-based transfers in a way that is transparent (ie. the end-user doesn’t need to know it’s proprietary since they don’t need to do anything special on their machine or pay a fee to use it).
But I ramble.
Spot on about business users vs consumer lives. Finland will make broadband a legal right come next July (http://bit.ly/Vtaup)
I like your MFT, which is very useful for files that get sent only once
For e.g. company updates, software updates, a peer-to-peer filesharing would be great. Direct Connect, a p2p sharing network similar to Napster, is now slowly moving to the ADC protocol (http://bit.ly/FBqZC)
It supports bzip compression, tiger tree hashing, and TLS, among others – fast, failfree and secure. It also supports simultaneous downloads where one file will be split across any number of users in order to minimise downloading time
You have to (usually) register yourself with a name and password to a hub, and the server itself doesn’t contain anything to share
It’s open, it’s free, and actively supported too. All you need is clients, and a very small server
Try http://sourceforge.net/search/?type_of_search=soft&words=adc to find soft releases (client or server) in any language