Bug #59

Download won't finish with strange server behaviour

Added by W3ird_N3rd almost 2 years ago. Updated over 1 year ago.

Status: Closed Start: 09/23/2010
Priority: Normal Due date:
Assigned to: Alguno % Done:

0%

Category: NNTPGrab Core
Target version: 0.7.0
Votes: 0

Description

I've downloaded an old post (200 days). The first 4,5GB would download fine, slowly, but fine. It appears that this post was perhaps not cached on the server. Downloading with lots of connections (where normally I can easily get full speed with 1-2 connections) appeared to help.

At the last ~450MB the download pretty much stopped. I now set the maximum number of connections (8 in my case). Every time I pressed the "pause" button twice it would download another 10-20MB. The first 5 connections would stay dead (stuck at 0% and disconnect after a while) and the other 3 would download 10-20MB or so (and then die like the first five). So after a while hitting pause I got to the last 1.3MB.

And that's where it got really ugly.. Five files (5x300KB posts, each post for a different file) were to be completed. But as I said.. The first five connections would stay dead. I could leave this on for hours but those last five files will never complete.

If I put something else in the queue that'll start downloading at full speed with 3 connections, still leaving those 5 dead.

Sure, the fault is with the usenetserver. But if nntpgrab would consider those 5x300KB as lost, create those 5 files (that would probably still offer 44/45 blocks) and start PAR.. Well. That would be a better use of time compared to having 5x300KB in the queue that will complete when pigs fly.

nntpgrab.log - waiting for pigs to take off (14.6 KB) W3ird_N3rd, 09/23/2010 06:26 pm

nntpgrab.log - new logfile, now with 0.6.9 (87 KB) W3ird_N3rd, 09/23/2010 10:48 pm

nntpgrab.log (63.5 KB) W3ird_N3rd, 09/23/2010 10:58 pm

History

#1 Updated by Alguno almost 2 years ago

  • Category set to NNTPGrab Core
  • Status changed from New to In Progress
  • Assigned to set to Alguno
  • Target version set to 0.7.0

In the logfile I see this message appearing multiple times:

2010-09-23 17:56:45 INFO NNTPGrab Core Request received to disconnect connection 18, disconnect_type = 6, reason = (null), cause = download_thread.c:367, recv_buf_len = 1, recv_buf=

Recently, some changes were done in this area (r1703) in order to prevent situations where connections might hang.
As you're using NNTPGrab 0.6.2 I recommend you to try again with NNTPGrab 0.6.90.

#2 Updated by W3ird_N3rd almost 2 years ago

  • File nntpgrab.log added

#3 Updated by W3ird_N3rd almost 2 years ago

  • File nntpgrab.log added

More 0.6.9 nntpgrab log. I loaded those 5 files that did not complete on 0.6.2 but they won't complete on 0.6.9 either. I've set 2 threads. They will download up to 99% and that remaining last post never arrives.

#4 Updated by Alguno over 1 year ago

  • Status changed from In Progress to Closed

Various stability issues have been fixed some the original report, so the mentioned behaviour should be resolved as well

Also available in: Atom PDF