Modify

Opened 9 years ago

Closed 6 years ago

Last modified 4 years ago

#4420 closed defect (fixed)

rtorrent shows error when checking single file bigger than 4GB

Reported by: vn158@… Owned by: developers
Priority: response-needed Milestone: Barrier Breaker 14.07
Component: packages Version: Trunk
Keywords: rtorrent torrent Cc:

Description

rtorrent downloads whole file but after final hashing it shows "Hash check on download completion found bad chunks, consider using safe_sync" I tested md5sum and file is broken.

I am using Asus wl500gP with Kamikaze 8.09RC1 (kernel 2.4) and drive with ext2 filesystem.

same bug is mentioned here: http://forum.openwrt.org/viewtopic.php?id=17803

Attachments (1)

800-mips-mmap2-enable.patch (1.2 KB) - added by highfly22@… 7 years ago.
patch for uclibc mmap2

Download all attachments as: .zip

Change History (26)

comment:1 Changed 9 years ago by vn158@…

I tried to check downloaded file with uTorrent and it shows begin and end of file is broken. It looks like number overflow somewhere in rtorrent code.

comment:2 Changed 9 years ago by nbd

  • Milestone changed from Kamikaze 8.09 RC2 to Kamikaze

comment:3 Changed 9 years ago by florian

  • Resolution set to fixed
  • Status changed from new to closed

Should be fixed with [15090].

comment:4 Changed 9 years ago by cozmic

I also ran into this problem with a 16523, so it's not fixed. It's probably not rtorrent/libtorrent's fault, but a problem in the mmap() handling in uclibc or the kernel.

comment:5 Changed 8 years ago by sln <slaninka@…>

I experience the same issue with 18260

comment:6 Changed 8 years ago by anonymous

Setting

# 3.5G
split_file_size = 3758096384

in rtorrent.rc helps. Only several broken chunks, easely fixable by redownloading (this is usual situation for almost any big file). Tested on r18732.
Also if moving complete download is configured, simple "mv" is not enough. I had to use a script like that:

#!/bin/sh
from=$1
to=$2
mv="mv -f"
#cut 3-digit suffix                                  
from_new="$(echo "$from" | sed -e 's/[0-9]\{3\}$//')"
#move all parts if suffix present                                                      
[ "$from" == "$from_new" ] && $mv "$from" "$to" || $mv "$from_new"[0-9][0-9][0-9] "$to"

comment:7 Changed 8 years ago by Zajec

I've installed "Backfire 10.03 Beta", then added kmod-usb2 and kmod-usb-storage plus following:
kmod-fs-ext4 - 2.6.32.9-1
rtorrent - 0.8.6_r1130-1
libtorrent - 0.12.6_r1130-2

There is partial output of my rtorrent console:

   Single
            done     1340.6 MB Rate:   3.4 /   0.0 KB Uploaded:  1352.5 MB                 [   R: 1.01]
  Tracker[8:8]: Connecting to udp://denis.stalker.h3q.com:6969/announce

   Up
  [OPEN]    4096.0 / 4476.8 MB Rate:   0.0 /   0.0 KB Uploaded:     0.0 MB                 [   R: 0.00]
  Inactive: Hash check on download completion found bad chunks, consider using "safe_sync".

   An
  [OPEN]    4094.1 / 4466.1 MB Rate:   0.0 /   0.0 KB Uploaded:     8.5 MB                 [   R: 0.00]
  Inactive: Hash check on download completion found bad chunks, consider using "safe_sync".

   Where
  [OPEN]    4156.0 / 6707.6 MB Rate:   0.0 /   0.0 KB Uploaded:     0.0 MB                 [   R: 0.00]
  Inactive: Hash check on download completion found bad chunks, consider using "safe_sync".

   Life
            done     1504.6 MB Rate:  46.5 /   0.0 KB Uploaded:   859.6 MB                 [   R: 0.57]
  Tracker: [Tried all trackers.]

As you can see, only big torrents (over 4GB) have problem. When I use same external (USB) disk with same rtorrent version (0.8.6) I do not see that errors.

There is also some discussion in bug report on rtorrent site: http://libtorrent.rakshasa.no/ticket/483

If this problem still exist, can we reopen it then?

comment:8 Changed 8 years ago by Zajec

I meant using same disk and same rtorrent version with my PC.

comment:9 Changed 8 years ago by Zajec

  • Resolution fixed deleted
  • Status changed from closed to reopened

I installed Backfire 10.03 final and it's still affected. It's for my USB driver with ext4 partition plugged to Linksys WRT160NL.

comment:10 Changed 8 years ago by anonymous

IIRC, there is a option somewhere in menuconfig to enable large file support >= 4GB...

comment:11 Changed 8 years ago by Zajec

I downloaded whole openSUSE-DVD-Build0625-i586.iso in OpenWRT's rtorrent and it was successfully downloaded only in 4096 MB (of 4117.8 MB). Then unmounted disk, connected it to my PC, used my PC's rtorrent. It downloaded the rest of file, checked hash sum and *it was fine*. So I unmounted disk again and connected it to OpenWRT again. OpenWRT's rtorrent checked hash sum again and it failed at 4096 MB again. I let it download the rest of file but it failed again (when checking hash sum).

It seems rtorrent completely fails on both: reading and writing part of file that is above 4096 MB.

I started suspecting OpenWRT's kernel configuration. anonymous from comment above makes me really believe in that. Could someone confirm this?

As reporter (vn158) hit this issue with ext2 (for me it's ext4) it would mean it's some not-specific-filesystem configure option.

comment:12 Changed 7 years ago by sniperpr@…

backfire default config is open Large files option.

i find this problem,can't hash > 4G file.
if your down file > 4G. rtorrent only hash 4G, more size, rtorrent must RE download.

first, On the pc. user uTorrent down >4G one file.
copy to my dir-825.
open rtorrent. only hash 4G size, more size ,must RE download.

how to fix it ?

thanks.

comment:13 follow-up: Changed 7 years ago by sniperpr

sorry.i forget put my hdd format type infomation.

my dir-825 use USB-HDD.
user reiserFS 3.6

comment:14 in reply to: ↑ 13 Changed 7 years ago by anonymous

seeing this issue (at 4096MB) also on a ppc405/uclibc machine. i don't have any ideas aside from the fact that expressed in bytes, 4096MB is the largest number that will fit into 32bit unsigned int. i guess the question then becomes, in which part of the code does that become a limiting factor? what about a uclibc/embedded system, in this regard, is different than an glibc/x86 system?

comment:15 Changed 7 years ago by jow

  • Version set to Trunk

There was a commit to transmission recently which defined _GNU_SOURCE. That made the code switch from pread/pwrite to pread64/pwrite64. Wouldn't surprise me if this issue here is exactly the same.

comment:16 Changed 7 years ago by anonymous

i've recompiled libtorrent/rtorrent with -D_GNU_SOURCE in the cxxflags. i'll update this with reports on success/failure of these changes.

comment:17 Changed 7 years ago by anonymous

unfortunately, rtorrent still is failing hash check at the same spot with GNU_SOURCE defined during build:

* [OPEN]    4096.0 / 4472.3 MB Rate:   0.0 /   0.0 KB Uploaded:    16.3 MB                 [   R: 0.00]
* Inactive: Hash check on download completion found bad chunks, consider using "safe_sync".

comment:18 Changed 7 years ago by nbd

  • Priority changed from normal to response-needed

Please try latest trunk or backfire rebuilt from scratch

comment:19 Changed 7 years ago by highfly22@…

I finally found this is a uclibc bug.
Please try the patch of uclibc

Changed 7 years ago by highfly22@…

patch for uclibc mmap2

comment:20 Changed 6 years ago by nbd

  • Resolution set to obsolete
  • Status changed from reopened to closed

that issue seems to have been fixed in uclibc upstream.

comment:21 Changed 6 years ago by khim

  • Resolution obsolete deleted
  • Status changed from closed to reopened

Issue is fixed in uClibc 0.9.33.1 but AFAICS the newest version of uClibc available for openwrt is 0.9.33 thus rtorrent still works incorrectly with files > 4GB in size...

comment:22 Changed 6 years ago by ryzhov_al

Can't confirm that. Rtorrent 4GB hash check works after uClibc team patched nmap2 code in all stable branches (https://github.com/rakshasa/rtorrent/issues/52).

Checked with rtorrent 0.9.2 + libtorrent 0.13.2 + uclibc 0.9.32 on my OpenWRT based repo (http://code.google.com/p/wl500g-repo/)

comment:23 Changed 6 years ago by khim

Ah-ha. I can not test with stable (my Buffalo WZR-HP-AG300H is only supported by trunk) but I've tested with trunk and can confirm that hand-compiled uClibc 0.9.33 with patch fixes the problem. Out-of-the-box it does not work :-(

comment:24 Changed 6 years ago by nbd

  • Resolution set to fixed
  • Status changed from reopened to closed

fixed in r31740

comment:25 Changed 4 years ago by jow

  • Milestone changed from Attitude Adjustment 12.09 to Barrier Breaker 14.07

Milestone Attitude Adjustment 12.09 deleted

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.