Why Downloads Fail (Part 2)



In Part 1, I defined some key terms and concepts about measuring download success rate. Now I'm ready to drill down into some details.

So, why do downloads fail?

Note that I'm using the word "fail" to mean "did not complete for whatever reason." I do not mean to limit the discussion to technical failures. It's especially important to make that distinction because, in fact, we believe  the number one reason is not technical -- the main reason seems to be users stop the download.

From the server side, we only know the stream of bits going to the customer was broken before the last bytes were served. We really can't tell if that was because something outside our control "broke" (technical issue) or the user simply hit "cancel". We suspected user actions played a large role but weren't sure how to "prove" this. While I'm not saying we did prove this  100% scientifically, here's some of our reasoning for the hypothesis provided.

First, while our infrastructure provides all the downloads on Sun's download center, the same infrastructure also supplies all the files for Java SE and JRE auto-update functionality. We noticed the completion rate for similarly sized files was higher for auto-update. Since the files are on the same infrastructure and of similar size, what would explain the difference? A plausible explanation is that once a user approves the auto-update, the downloading happens automatically and with no readily apparent way for a user to stop the download once it's started. This would seem to indicate that the more automated you make a download, and the less chance a customer has to cancel it, the higher the success rate.

Next, we had a really large file (400+ MB) with a success rate of under 50%. Similarly sized files were running 65-75%, which really is very good for such huge files (based on our experience as well as some benchmarking with other vendors). The product team with this particular file kept suggesting there might be something wrong with our system, so we tried to dispel that. We took two specific actions:
  1. We set up a large file synthetic transaction (LFST). A synthetic transaction is an automated, scripted web transaction that mimics exactly the steps a user takes to download a file. We put the same problematic file on public-facing servers in the US, Holland, and Singapore. Our LFST then ran every 3 hours 24x7 against each location -- it logged in to the site, accepted the license, and downloaded the file, exactly as a "real" human would do. The results were enlightening. The success rate was greater than 99% in the US and varied (over time) in the other locations from a low in the mid 80's to an average in the high 90s. These downloads traveled over the Internet to these locations from the download center, just as if end customers were downloading. So if a "robot" gets 85%-100% success rate but the public achieves 50%, isn't the most likely explanation that user's abort the download? (In some cases, we know users may have infrastructure issues not associated with our test sites, but frankly our sites weren't that well-connected outside the US, using generally highly-taxed Internet pipes out of  smaller data centers.)
  2. For further confirmation, we sent a survey to customers who had downloaded (or tried to download) this product and asked them about their experience. Note surveys can skew toward those who are dissatisfied and want to tell you about it, but even with that taken into account, there were very few problems reported. Most users said they got the file  no problem. If technical issues were contributing to a 50+% failure rate, I think we would've heard it loud and clear. Instead, customers were overwhelmingly "satisfied" or "very satisfied" with their download experience.

Of course there are technical problems that cause downloads to fail (I'll get to some of those in "part 3"), and "blaming" our customers is not a solution. But these findings seem to validate that the main reason downloads fail (for us, at least) is that users cancel their downloads before they complete. While the information is valuable,  I'm certainly not saying this is a good thing! We need to continue to understand why this happens and how to mitigate it, as we're always trying to improve completion rates and customer satisfaction. (And I'll get into some ideas on how to do that as well at a later date). But we do believe these findings indicate we don't have any systemic technical issues with our download infrastructures and applications, and that is a good thing.



Comments:

The only time that I've cancelled a download was when it was running extremely slowly. After 12 hours, I got tired of waiting. This hasn't happened recently. The other problem that I've encountered is that sometimes I download one or two CD images, and then when I click on the next one, I get a page that says `file not found'. I've never been able to recover from that condition, other than starting the whole process over again and downloading the remaining images.

Posted by Gary Mills on November 17, 2005 at 09:57 AM PST #

Gary, thanks for your feedback. I suspect "it was running extremely slowly" is one of the main reasons people cancel downloads so am not surprised to hear you say that! It's usually pretty hard to say where the bottleneck is occurring however.

Regarding the "file not found" issue, if that's happening on the Sun Download Center, (obviously) it shouldn't be. We'd appreciate it if you could help us debug the issue. Please copy the failing URL from the web page and send it to us using our customer feedback form. Just note the failure you're seeing and that'll really help us figure out what's going on.

Posted by Gary Zellerbach on November 18, 2005 at 12:40 AM PST #

Thanks for your great feed. But,I`ll eager to know about the Embargo country? From JSE5 We can`t download Java at all.(I mean we can`t download it form SUN and Related sites) why sun decide to do this??How this Policy implemented? Thanks

Posted by Hossein on November 19, 2005 at 07:15 PM PST #

Hi Hossein,
Sorry but I really can't go into details about export controls other than confirming that they exist. We use them to comply with regulations set by the US Government, so this is certainly beyond my control. The best thing I can do is point you to our FAQ on the subject. From there, you can contact our Customer Support team if you have further questions.

You do raise an interesting point -- another reason downloads fail is because we block them. I will try to comment further on that when I have time for another post. Thanks.

Posted by Gary Zellerbach on November 21, 2005 at 05:12 AM PST #

Hi Garry: Thank you so much man.Really I decided to make my own website and talk about this.I think this is exactly vice versa of what the WORLD called OpenSource.I think the US government policy is against the OpenSource Philosophy and this is contradiction. Anyway as you say,now we can do nothing,just we can sit and look at what`s going on by our politician :):) (But I am going to make my site and take look around SUN) :D anyway thanks Good Luck man

Posted by Hossein on November 21, 2005 at 05:29 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

I helped design, build, and manage download systems at Sun for many years. Recently I've focused on web eMarketing systems. Occasionally, I write about other interests, such as holography and jazz guitar. Follow me on Twitter: http://twitter.com/garyzel

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
News

No bookmarks in folder

Blogroll
ESD