Modify

Opened 5 years ago

Closed 5 years ago

Last modified 3 years ago

#12454 closed defect (fixed)

r34106 (stamp files for build variants) broke dnsmasq package's dhcpv6 variant

Reported by: hnyman Owned by: developers
Priority: response-needed Milestone: Chaos Calmer 15.05
Component: base system Version: Trunk
Keywords: dnsmasq ipv6 dhcpv6 Cc: hannu.nyman@…

Description

Changeset r34106 (isolating stamp files for build variants) has apparently broken dnsmasq package's dhcpv6 variant. It looks like including dnsmasq-dhcpv6 variant after r34106 causes the router not to boot up.

The firmware gets built normally and there is no apparent fault in the build process. But the router does not start after the flash. At least my ar71xx/wndr3700 does not. I had to de-brick it with a TFTP flash to get it back to running order.

dnsmasq's dhcpv6 variant got introduced by r32764. dnsmasq-dhcpv6 is visible in menuconfig next to normal plain dnsmasq.

During the past few months I have had in my config the normal dnsmasq disabled and dnsmasq-dhcpv6 enabled. And that has worked ok.

After r34106 that combination stops the router from booting. I have also tried having both enabled, but that does not work either.

I have spent this day trying to isolate the problem, but it took me a while and a few TFTP de-brickings... If you first build a pre-34106 version and do then svn up to post-r34106 and build without a "make clean", the build is normal and boots. Apparently the previous build's leftovers are enough to get included in the final firmware.

But if you "make clean" and build then a post-r34106 version, the build succeeds but is not bootable.

dnsmasq-dhcpv6 looks a bit strange compared to most packages, as it tries to be a variant instead of being an add-on toggle to decide on inclusion of dhcpv6 functionality in the main package.

Attachments (1)

WNDR3700-trunk-r34105-2012-11-11-1841.diffconfig (4.5 KB) - added by hnyman <hannu.nyman@…> 5 years ago.
Diffconfig

Download all attachments as: .zip

Change History (17)

comment:1 Changed 5 years ago by jow

  • Priority changed from normal to response-needed

I really fail to see how a broken dnsmasq-dhcpv6 would prevent the router from booting. Please provide a complete bootlog.

comment:2 Changed 5 years ago by hnyman <hannu.nyman@…>

When the router "did not boot", it started the boot process but then got stuck in the middle, just the power led slowly blinking. I was unable to connect to the router with telnet or SSH, so there is no bootlog available.

comment:3 Changed 5 years ago by jogo

Can you please attach the output of scripts/diffconfig.sh of a config that triggers this problem?

Changed 5 years ago by hnyman <hannu.nyman@…>

Diffconfig

comment:4 Changed 5 years ago by hnyman <hannu.nyman@…>

I have built today several builds with identical config. (diffconfig added to the ticket)

Some of the builds have worked, some have not, even regarding version 34105. I have been unable to determine a clear reason to create failing/booting images, and I have used the vanilla dnsmasq today (with no dhcpv6 support). Jow is probably right that it is not about that package.

When the boot fails, the router answers to ping at 192.168.1.1, but does not answer to any connection attempt with telnet or ssh. And wireshark does not show the UDP packages with failsafe trigger messages. I guess that router somehow does not boot properly from u-boot to kernel, or something like that.

If nobody else starts complaining about similar issues, this might be something specific to my buildhost :-( My build host is Ubuntu 12.04 x64 in Virtualbox, but I have used the same machine to build Openwrt since April 4-5 times per week, so it sounds strange that it would have broken down now. Alternatively, the new Virtualbox version 4.2.4 screws things up.)

several build logs can be found in http://koti.welho.com/hnyman1/Openwrt/trunk_error_does_not_boot/

comment:5 follow-up: Changed 5 years ago by jogo

It would be helpful if you could attach serial and capture the bootlog of the device failing to boot.

comment:6 in reply to: ↑ 5 Changed 5 years ago by hnyman

Replying to jogo:

It would be helpful if you could attach serial and capture the bootlog of the device failing to boot.

I ordered a serial-USB converter, so in a few days I may have more info.

I am thinking about alternative explanations, and the first that comes into mind is the current ar71xx/wndr3700 kernel size optimization in the build process. After r29406 an "estimation version" of the kernel is built first, its size is assessed and then the final kernel is built with the size set by the estimation.

My kernel size is currently quite at a 64 kB border (917117, 916862 B = 13,994 x 64kB), and I am thinking if it is possible that the estimation is a few bytes short and the final kernel image just crosses the border by a few bytes and thus the kernel bootline gets set wrong.

https://dev.openwrt.org/browser/trunk/target/linux/ar71xx/image/Makefile#L571

The key line there is:

let 'kk = (((s + c) / (64 * 1024) + 1) * 64)';

which sets the allocated kernel size. It sets the size based on the estimation size, it first calculates the used 64k blocks and rounds upward to the next 64 kB size. As my kernel is 13.994 x 64k, no buffer gets actually added as (13.994 gets rounded down to 13, then 1 is added, and 14.00 is the final about of 64k blocks used.)

One supporting factor for this line of reasoning is that there was one firmware version, the first with which I noticed this problem, where I had succesfully flashed my 3700v1, but that then failed in a 3700v2, although it had been built on the same compilation run. I threw that away so I can't verify my assumption.

I will attempt to increase the buffer set in Makefile and try rebuilding tonight.

Probably with something like:

let 'kk = (((s + c + 4096) / (64 * 1024) + 1) * 64)';

comment:7 Changed 5 years ago by hnyman <hannu.nyman@…>

It looks like it is not the kernel size. I checked some of the the "bad builds" with a hex editor, but there are a few dozens of 00 bytes at the end of the kernel area, so kernel size overflow is not the reason. I also experimented adding a few kilobytes to the size calculation, but no effect.

My last build 34174 runs ok in 3700v2, but does not boot in 3700v1. The versions are built during the same build run, so they should be as identical as possible. And that booting difference sounds strange.

I have uploaded the build r34174-2012-11-12-2328 to http://koti.welho.com/hnyman1/Openwrt/trunk_error_does_not_boot/ , so if somebody wants to test the build with a v1, please do so.

Could a bad flash chip be a reason? So that some bit does not get toggled by flashing any more, and it randomly depends on the firmware image, if the bit is right. I have flashed both routers over 100 times, but that should not be too much for the flash chip, should it? Is there any good way to check the flash quality? Reading whole flash contents from /dev/mtd and comparing to the original image? Anything better?

(Alternative reason might be something wrong in image generation.)

When I receive the serial port converter, I will test my 3700v1 and see the boot log what it says. Hopefully that will give some insights.

comment:8 Changed 5 years ago by anonymous

I have now a serial to USB converter and the bootlog from serial console is as below. The key part is probably:

   Verifying Checksum ... OK
### SQUASHFS loading 'image/uImage' to 0x80800000
lzma_fs returned unexpected result 0x1
SQUASHFS error: squashfs_readdir: read_block
### SQUASHFS LOAD ERROR<0> for image!

I earlier thought that it might be faulty flash memory, but at the first glance that does not sound like that, as checksum is right, but lzma_fs returns error regarding squashfs.

Any thoughts?

Full serial console bootlog:

64 MB
Top of RAM usable for U-Boot at: 84000000
Reserving 315k for U-Boot at: 83fb0000
Reserving 192k for malloc() at: 83f80000
Reserving 44 Bytes for Board Info at: 83f7ffd4
Reserving 36 Bytes for Global Data at: 83f7ffb0
Reserving 128k for boot params() at: 83f5ffb0
Stack Pointer at: 83f5ff98
Now running in RAM - U-Boot at: 83fb0000
id read 0x100000ff
flash size 8MB, sector count = 128
Flash:  8 MB
In:    serial
Out:   serial
Err:   serial
Net:   ag7100_enet_initialize...
CHH:mac: 0 if: 2
CHH:mac:verify: 0 if: 00000002
: cfg1 0xf cfg2 0x7014
eth0: 30:46:9a:1b:4d:36
eth0 up
CHH:mac: 1 if: 1
CHH:mac:verify: 1 if: 00000001
: cfg1 0xf cfg2 0x7014
eth1: 30:46:9a:1b:4d:37
eth1 up
eth0, eth1
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

 Client starts...[Listening] for ADVERTISE...TTT
Retry count exceeded; boot the image as usual

 nmrp server is stopped or failed !
Hit any key to stop autoboot:  0
   Verifying Checksum ... OK
### SQUASHFS loading 'image/uImage' to 0x80800000
lzma_fs returned unexpected result 0x1
SQUASHFS error: squashfs_readdir: read_block
### SQUASHFS LOAD ERROR<0> for image!
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

The Router is in TFTP Server Firmware Recovery mode NOW!
Listening on Port : 69, IP Address: 192.168.1.1...
Upgrade Mode

comment:9 Changed 5 years ago by hnyman <hannu.nyman@…>

Full bootlog again includding the few first lines, and my correct name...

U-Boot 1.1.4DNI1.6 (May 22 2009 - 16:37:44)

WNDR3700U (ar7100) U-boot 0.0.12
DRAM:  b8050000: 0xc0140180
64 MB
Top of RAM usable for U-Boot at: 84000000
Reserving 315k for U-Boot at: 83fb0000
Reserving 192k for malloc() at: 83f80000
Reserving 44 Bytes for Board Info at: 83f7ffd4
Reserving 36 Bytes for Global Data at: 83f7ffb0
Reserving 128k for boot params() at: 83f5ffb0
Stack Pointer at: 83f5ff98
Now running in RAM - U-Boot at: 83fb0000
id read 0x100000ff
flash size 8MB, sector count = 128
Flash:  8 MB
In:    serial
Out:   serial
Err:   serial
Net:   ag7100_enet_initialize...
CHH:mac: 0 if: 2
CHH:mac:verify: 0 if: 00000002
: cfg1 0xf cfg2 0x7014
eth0: 30:46:9a:1b:4d:36
eth0 up
CHH:mac: 1 if: 1
CHH:mac:verify: 1 if: 00000001
: cfg1 0xf cfg2 0x7014
eth1: 30:46:9a:1b:4d:37
eth1 up
eth0, eth1
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

 Client starts...[Listening] for ADVERTISE...TTT
Retry count exceeded; boot the image as usual

 nmrp server is stopped or failed !
Hit any key to stop autoboot:  0
   Verifying Checksum ... OK
### SQUASHFS loading 'image/uImage' to 0x80800000
lzma_fs returned unexpected result 0x1
SQUASHFS error: squashfs_readdir: read_block
### SQUASHFS LOAD ERROR<0> for image!
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

The Router is in TFTP Server Firmware Recovery mode NOW!
Listening on Port : 69, IP Address: 192.168.1.1...
Upgrade Mode

comment:10 Changed 5 years ago by hnyman <hannu.nyman@…>

Additional info:
I stopped the u-boot boot process with keyboard and tried a few commands in u-boot. I have no experience with u-boot, so I am wondering what these results mean...

Retry count exceeded; boot the image as usual

 nmrp server is stopped or failed !
Hit any key to stop autoboot:  0
ar7100> printenv
bootdelay=4
baudrate=115200
bootargs=console=ttyS0,115200 root=31:09 rootfstype=squashfs init=/etc/preinit mtdparts=ar7100-nor0:320k(uboot),128k(env),7296k(rootfs),64k(config),64k(config_bak),64k(pot),64k(traffic_meter),128k(debug_info),64k(caldata),7471040@458816(mount_fs)
filesize=450a0
fileaddr=80010000
ipaddr=192.168.1.2
serverip=192.168.1.12
bootcmd=fsload 80800000 image/uImage;bootm 80800000
stdin=serial
stdout=serial
stderr=serial
ethact=eth0

Environment size: 454/65532 bytes
ar7100> iminfo 80800000

## Checking Image at 80800000 ...
   Bad Magic Number
ar7100>

comment:11 Changed 5 years ago by hnyman <hannu.nyman@…>

The strange thing is that I can flash again those firmware images that I know from earlier experience to be working. Below is the serial console log of a successful recovery TFTP transfer. The boot process after that looks quite similar to the failing ones until the point where squashfs image is successfully decompressed to 0x80800000 and then control gets transferred to Linux kernel:

### SQUASHFS loading 'image/uImage' to 0x80800000
### SQUASHFS load complete: 916861 bytes loaded to 0x80800000
## Booting image at 80800000 ...
   Image Name:   MIPS OpenWrt Linux-3.3.8
   Created:      2012-11-05  19:35:33 UTC
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    916797 Bytes = 895.3 kB
   Load Address: 80060000
   Entry Point:  80060000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80060000) ...
## Giving linux memsize in bytes, 67108864

Starting kernel ...

[    0.000000] Linux version 3.3.8 (perus@hnvb) (gcc version 4.6.4 20121001 (prerelease) (Linaro GCC 4.6-2012.10) ) #2 Mon Nov 5 21:34:44 EET 2012
[    0.000000] bootconsole [early0] enabled

Somehow I find it hard to believe that is a flash failure. Somehow it sounds like there is an illegal structure in the squashfs compressed image that breaks the decompression, or something like that.

More context:

#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

 Client starts...[Listening] for ADVERTISE...checksum bad
Tchecksum bad
checksum bad
Tchecksum bad
T
Retry count exceeded; boot the image as usual

 nmrp server is stopped or failed !
Hit any key to stop autoboot:  0
   Verifying Checksum ... OK
### SQUASHFS loading 'image/uImage' to 0x80800000
lzma_fs returned unexpected result 0x1
SQUASHFS error: squashfs_readdir: read_block
### SQUASHFS LOAD ERROR<0> for image!
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0x2fffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

The Router is in TFTP Server Firmware Recovery mode NOW!
Listening on Port : 69, IP Address: 192.168.1.1...

Rcv:
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        .................................................................
        ..................................................
Done!
Bytes transferred = 5243013 (500085 hex)
Erase 7 - 120 sectors...
First 0x7 last 0x78 sector size 0x10000                                      120
Copy image to Flash... write addr: bf070000
Done
Rebooting...

Resetting...þ

U-Boot 1.1.4DNI1.6 (May 22 2009 - 16:37:44)

WNDR3700U (ar7100) U-boot 0.0.12
DRAM:  b8050000: 0xc0140180
64 MB
Top of RAM usable for U-Boot at: 84000000
Reserving 315k for U-Boot at: 83fb0000
Reserving 192k for malloc() at: 83f80000
Reserving 44 Bytes for Board Info at: 83f7ffd4
Reserving 36 Bytes for Global Data at: 83f7ffb0
Reserving 128k for boot params() at: 83f5ffb0
Stack Pointer at: 83f5ff98
Now running in RAM - U-Boot at: 83fb0000
id read 0x100000ff
flash size 8MB, sector count = 128
Flash:  8 MB
In:    serial
Out:   serial
Err:   serial
Net:   ag7100_enet_initialize...
CHH:mac: 0 if: 2
CHH:mac:verify: 0 if: 00000002
: cfg1 0xf cfg2 0x7014
eth0: 30:46:9a:1b:4d:36
eth0 up
CHH:mac: 1 if: 1
CHH:mac:verify: 1 if: 00000001
: cfg1 0xf cfg2 0x7014
eth1: 30:46:9a:1b:4d:37
eth1 up
eth0, eth1
Trying eth0
: unit 0 phy is up...RGMii 1000Mbps full duplex
#259:ag7100_set_mac_from_link
: pll reg 0x18050010: 0x11110000
: cfg_1: 0x1ff0000
: cfg_2: 0x3ff
: cfg_3: 0x8001ff
: cfg_4: 0xffff
: cfg_5: 0xfffef
: done cfg2 0x7215 ifctl 0x40605060 miictrl 0x22

 Client starts...[Listening] for ADVERTISE...checksum bad
Tchecksum bad
checksum bad
Tchecksum bad
T
Retry count exceeded; boot the image as usual

 nmrp server is stopped or failed !
Hit any key to stop autoboot:  0
   Verifying Checksum ... OK
### SQUASHFS loading 'image/uImage' to 0x80800000
### SQUASHFS load complete: 916861 bytes loaded to 0x80800000
## Booting image at 80800000 ...
   Image Name:   MIPS OpenWrt Linux-3.3.8
   Created:      2012-11-05  19:35:33 UTC
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    916797 Bytes = 895.3 kB
   Load Address: 80060000
   Entry Point:  80060000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80060000) ...
## Giving linux memsize in bytes, 67108864

Starting kernel ...

[    0.000000] Linux version 3.3.8 (perus@hnvb) (gcc version 4.6.4 20121001 (prerelease) (Linaro GCC 4.6-2012.10) ) #2 Mon Nov 5 21:34:44 EET 2012
[    0.000000] bootconsole [early0] enabled
[    0.000000] CPU revision is: 00019374 (MIPS 24Kc)
[    0.000000] SoC: Atheros AR7161 rev 2
[    0.000000] Clocks: CPU:680.000MHz, DDR:340.000MHz, AHB:170.000MHz, Ref:40.000MHz
[    0.000000] Determined physical RAM map:
[    0.000000]  memory: 04000000 @ 00000000 (usable)
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] Zone PFN ranges:
[    0.000000]   Normal   0x00000000 -> 0x00004000
[    0.000000] Movable zone start PFN for each node
[    0.000000] Early memory PFN ranges
[    0.000000]     0: 0x00000000 -> 0x00004000
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
[    0.000000] Kernel command line:  board=WNDR3700 console=ttyS0,115200 mtdparts=spi0.0:320k(u-boot)ro,128k(u-boot-env)ro,896k(kernel),6784k(rootfs),64k(art)ro,7680k@0x70000(firmware) rootfstype=squashfs,jffs2 noinitrd
[    0.000000] PID hash table entries: 256 (order: -2, 1024 bytes)

comment:12 Changed 5 years ago by hnyman <hannu.nyman@…>

This ticket should maybe closed as invalid and a new one opened, as the problem description is wrong. The problem is there for several WNDR3700/3800 users, but it is probably something related to invalid lzma compressed images instead of package stamps.

See discussion at openwrt-devel and forum:
https://lists.openwrt.org/pipermail/openwrt-devel/2012-November/thread.html#17445
https://forum.openwrt.org/viewtopic.php?id=40565

On 18.11.2012 5:55, Adam Gensler wrote:

Verifying Checksum ... OK

### SQUASHFS loading 'image/uImage' to 0x80800000
ERROR: LzmaDecode.c, 547
lzma_fs returned unexpected result 0x1
SQUASHFS error: squashfs_readdir: read_block
### SQUASHFS LOAD ERROR<0> for image!

I fetched wndr3800 sources from Netgear and extracted the u-boot part here:
http://koti.welho.com/hnyman1/Openwrt/trunk_error_does_not_boot/Netgear3800_uboot.zip

At the first glance it looks like the lib_bootstarp/lzmadecode.c basicly
returns 0 for success and 1 for most errors. There are several places where
errors are returned, but in most cases just LZMA_RESULT_DATA_ERROR is
returned, and that is 1. So no clue from the error number.

Based on the line number 547, the error is probably coming quite from the end
of the source file, possibly from this function:

       len += kMatchMinLen;
       #ifdef _LZMA_OUT_READ
       if (rep0 > distanceLimit)
       #else
       if (rep0 > nowPos)
       #endif
       {

#ifdef DEBUG_ENABLE_BOOTSTRAP_PRINTF
         printf("ERROR: %s, %d\n", __FILE__, __LINE__);
#endif
         return LZMA_RESULT_DATA_ERROR;
       }

That would suggest that the error is related to the structure of the
generated lzma encoded file.

comment:13 Changed 5 years ago by hnyman <hannu.nyman@…>

One idea is the lzma parameters " -lc1 -lp2 -pb2 ", which have been introduced without any explanation four years ago with r12628. Those differ from the defaults mentioned in the lzma's built-in help texts, and do differ from Netgear's build process (which uses the defaults in lzma and version 4.32, not 4.65).

http://nbd.name/gitweb.cgi?p=openwrt.git;a=commitdiff;h=6fb33f05da8ff895875cb8d98a7c18e9c606e205

It might be theoretically possible, that the parameter create a lzma version that the u-boot chokes on. Why should that surface then now, is then more problematic to answer...

That was made 4 years ago, so it is really old, but is a deviation from the Netgear's own routines, shown in forum: https://forum.openwrt.org/viewtopic.php?pid=183591#p183591

I built 34036 and 34066 fully from scratch and they both worked both in v1 and v2.
Then I built 34085 and it failed in both routers.

I then modified the ar71xx image makefile and removed those non-standard compression parameters from the lzma command line. And a new 34085 then booted ok in both routers.

-- trunk/target/linux/ar71xx/image/Makefile    (revision 34085)
+++ trunk/target/linux/ar71xx/image/Makefile    (working copy)
@@ -69,7 +69,7 @@
endif

define CompressLzma
-  $(STAGING_DIR_HOST)/bin/lzma e $(1) -lc1 -lp2 -pb2 $(2)
+  $(STAGING_DIR_HOST)/bin/lzma e $(1) $(2)
endef

define PatchKernelLzma

I will rebuild 34085 once again from scratch and retry. (I made the last build without clean, as it was supposed change just the image creation...)

Read about the lzma/xz memory consumption here:
http://linux.die.net/man/1/lzma
http://sourceforge.net/projects/lzmautils/forums/forum/708858/topic/3965542

Based on those articles, it might be sensible to either revert to default parameters, or based on seeing the parameters at staging_dir/host/bin/lzma to change the default directory size from 23 bits (8 MB) to something smaller, as the firmware is usually much smaller. (see below from lzma help file.)

perus@ubvb:/Openwrt/trunk$ staging_dir/host/bin/lzma 

LZMA 4.65 : Igor Pavlov : Public domain : 2009-02-03

Usage:  LZMA <e|d> inputFile outputFile [<switches>...]
  e: encode file
  d: decode file
  b: Benchmark
<Switches>
  -a{N}:  set compression mode - [0, 1], default: 1 (max)
  -d{N}:  set dictionary size - [12, 30], default: 23 (8MB)
  -fb{N}: set number of fast bytes - [5, 273], default: 128
  -mc{N}: set number of cycles for match finder
  -lc{N}: set number of literal context bits - [0, 8], default: 3
  -lp{N}: set number of literal pos bits - [0, 4], default: 0
  -pb{N}: set number of pos bits - [0, 4], default: 2
  -mf{MF_ID}: set Match Finder: [bt2, bt3, bt4, hc4], default: bt4
  -mt{N}: set number of CPU threads
  -eos:   write End Of Stream marker
  -si:    read data from stdin
  -so:    write data to stdout
dict=size
    Dictionary (history buffer) size indicates how many bytes of the recently processed uncompressed data is kept in memory. One method to reduce size of the uncompressed data is to store distance-length pairs, which indicate what data to repeat from the dictionary buffer. The bigger the dictionary, the better the compression ratio usually is, but dictionaries bigger than the uncompressed data are waste of RAM. 
Typical dictionary size is from 64 KiB to 64 MiB. The minimum is 4 KiB.
    The maximum for compression is currently 1.5 GiB. The decompressor already supports dictionaries up to one byte less than 4 GiB, which is the maximum for LZMA1 and LZMA2 stream formats. 
Dictionary size has the biggest effect on compression ratio.
    Dictionary size and match finder together determine the memory usage of the LZMA1 or LZMA2 encoder. The same dictionary size is required for decompressing that was used when compressing, thus the memory usage of the decoder is determined by the dictionary size used when compressing.

comment:14 Changed 5 years ago by hnyman <hannu.nyman@…>

Looks like the reason was indeed the lzma compression parameters.

I have already built newest trunk 34245 and it works both in my v1 and v2 routers.

I actually created the images with two different parameter lines in target/linux/ar71xx/image/Makefile:

First that already discussed removal of the current parameters:

$(STAGING_DIR_HOST)/bin/lzma e $(1) $(2)

But then also with original parameters, but added the directory size limitation from the default 8MB to 1 MB = 20 bits.

$(STAGING_DIR_HOST)/bin/lzma e $(1) -lc1 -lp2 -pb2 -d20 $(2)

Both versions worked in both of my routers.

If nobody suggests anything better, I will continue to live with this change:

--- trunk/target/linux/ar71xx/image/Makefile    (revision 34245)
+++ trunk/target/linux/ar71xx/image/Makefile    (working copy)
@@ -69,7 +69,7 @@
 endif

 define CompressLzma
-  $(STAGING_DIR_HOST)/bin/lzma e $(1) -lc1 -lp2 -pb2 $(2)
+  $(STAGING_DIR_HOST)/bin/lzma e $(1) -lc1 -lp2 -pb2 -d20 $(2)
 endef

 define PatchKernelLzma

comment:15 Changed 5 years ago by hnyman <hannu.nyman@…>

Fixed in trunk with r34248 which applies the 20-bit dictionary size option "-d20" to just wndr3700/3800 instead of all ar71xx images. Dictionary size will stay at the default 23 bits for the other routers.

(according to juhosg, this will probably be later fixed in AA, after the rc1 images have been built.)

comment:16 Changed 5 years ago by juhosg

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r34248.

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.