Modify

Opened 8 years ago

Closed 8 years ago

Last modified 4 years ago

#7676 closed defect (fixed)

ixp4xx config-2.6.32 needs to be resysnced

Reported by: Mark Kelly <mekelly5@…> Owned by: developers
Priority: highest Milestone: Barrier Breaker 14.07
Component: base system Version: Trunk
Keywords: Cc:

Description

Current trunk builds of ixp4xx jffs2 .img files are huge! lzma kernel config updates do not appear to be completely synced in the current config.

Temporarily you can switch over to 2.6.33 for the kernel and the image files are back to normal sizes.

Attachments (0)

Change History (17)

comment:1 Changed 8 years ago by anonymous

If I understand this correctly--I'm a bit of an OpenWRT noob--I'm seeing the same thing with my WNDR3700 builds (ar71xx platform, also 2.6.32 kernel currently). I haven't quite isolated the gzip ---> LZMA switchover for JFFS as the cause of the sudden bloat, but it's definitely somewhere in that timeframe (+/- 10 revs or so). For my config the difference is something like 3.8 MB vs. 6.0-6.5 MB.

comment:2 Changed 8 years ago by acoul

  • Resolution set to worksforme
  • Status changed from new to closed

as of trunk r22374 jffs2/lzma has been activated for 2.6.32.x kernels too.

comment:3 Changed 8 years ago by Mark Kelly <mekelly5@…>

  • Resolution worksforme deleted
  • Status changed from closed to reopened

I'm sorry but it's not working in 2.6.32 for ixp4xx as I said before. Picking 2.6.32 my images files are more than double the size of the image files by picking 2.6.33, so jffs2/lzma flags have not been set correctly in 2.6.32. My typical img is 12 megs, picking 2.6.32 I get 28 MB img files on a clean checkout of trunk of the exact same config.

comment:4 Changed 8 years ago by anonymous

Yes, it's r22374 that seems to be the problem. Notice how all the images in http://downloads.openwrt.org/snapshots/trunk/ar71xx/ were rebuilt earlier today (last having been rebuilt five days ago, a few hours before r22374), and if I'm not mistaken all the JFFS ones got at least 2 MB bigger.

Later tonight I should be able to build r22373 and r22374 with the same .config so it's easier to see the problem.

comment:5 Changed 8 years ago by anonymous

Okay, I actually decided to build the latest trunk (r22414) with the default config for the WNDR3700, both as-is and with r22374 reverted.

Here's the result:

$ svn co svn://svn.openwrt.org/openwrt/trunk
Checked out revision 22414.
$ mv trunk trunk22414
$ cp -R trunk22414{,m22374}
$ svn merge -c -22374 svn://svn.openwrt.org/openwrt/trunk trunk22414m22374/
--- Reverse-merging r22374 into 'trunk22414m22374':
U    trunk22414m22374/target/linux/generic/config-2.6.32
$ cd trunk22414
$ make menuconfig
# Target System AR71xx/AR7240/AR913x, Target Profile NETGEAR WNDR3700; no other changes
$ make
$ cd ../trunk22414m22374/
$ make menuconfig
# Target System AR71xx/AR7240/AR913x, Target Profile NETGEAR WNDR3700; no other changes
$ make
$ cd ..
$ cat sizes.sh
#!/bin/bash
old=trunk22414
new=trunk22414m22374
path=bin/ar71xx

for i in `ls $old/$path/*jffs*`; do
    du -h {$old,$new}/$path/`basename $i`
    echo
done

$ ./sizes.sh 
4.5M	trunk22414/bin/ar71xx/openwrt-ar71xx-generic-root.jffs2-128k
2.5M	trunk22414m22374/bin/ar71xx/openwrt-ar71xx-generic-root.jffs2-128k

4.4M	trunk22414/bin/ar71xx/openwrt-ar71xx-generic-root.jffs2-64k
2.5M	trunk22414m22374/bin/ar71xx/openwrt-ar71xx-generic-root.jffs2-64k

5.4M	trunk22414/bin/ar71xx/openwrt-ar71xx-generic-wndr3700-jffs2-factory.img
3.5M	trunk22414m22374/bin/ar71xx/openwrt-ar71xx-generic-wndr3700-jffs2-factory.img

5.4M	trunk22414/bin/ar71xx/openwrt-ar71xx-generic-wndr3700-jffs2-sysupgrade.bin
3.5M	trunk22414m22374/bin/ar71xx/openwrt-ar71xx-generic-wndr3700-jffs2-sysupgrade.bin

$

Something is obviously wrong here....

comment:6 Changed 8 years ago by acoul

please provide the following output from your 2.6.32.x build:

Compression mode: size
Compressors:
      none             compr: 22 blocks (21577)  decompr: 0 blocks
      zlib (prio:80) - compr: 0 blocks (0/0)  decompr: 0 blocks 
      lzma (prio:70) + compr: 581 blocks (922750/1974777)  decompr: 0 blocks 
     rtime (prio:50) + compr: 0 blocks (0/0)  decompr: 0 blocks 
Compression errors: 0

it's a few (couple) of lines before the end.

comment:7 Changed 8 years ago by anonymous

Here is the output for r22414. It'll be at least 12-14 hours before I can get back to r22414-without-r22374. Note that I'm leaving out the irrelevant path prefix:

trunk22414/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -x zlib -x lzma -e 64KiB -o trunk22414/build_dir/linux-ar71xx_generic/root.jffs2-64k -d trunk22414/build_dir/target-mips_r2_uClibc-0.9.30.1/root-ar71xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 1057 blocks (3323364)  decompr: 0 blocks
      zlib (prio:80) - compr: 0 blocks (0/0)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 408 blocks (1029674/1373446)  decompr: 0 blocks 
Compression errors: 0

[...]

trunk22414/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -x zlib -x lzma -e 128KiB -o trunk22414/build_dir/linux-ar71xx_generic/root.jffs2-128k -d trunk22414/build_dir/target-mips_r2_uClibc-0.9.30.1/root-ar71xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 1025 blocks (3313046)  decompr: 0 blocks
      zlib (prio:80) - compr: 0 blocks (0/0)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 412 blocks (1041172/1383764)  decompr: 0 blocks 
Compression errors: 0

comment:8 Changed 8 years ago by Mark Kelly <mekelly5@…>

Here's the output for R22417 for IXP4xx:

/opt/openwrt/k1/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -x zlib -x lzma -e 64KiB -o /opt/openwrt/k1/build_dir/linux-ixp4xx_generic/root.jffs2-64k -d /opt/openwrt/k1/build_dir/target-armeb_v5te_uClibc-0.9.30.3_eabi/root-ixp4xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 6992 blocks (23061848)  decompr: 0 blocks
      zlib (prio:80) - compr: 0 blocks (0/0)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 2502 blocks (6285636/8425846)  decompr: 0 blocks 
Compression errors: 0
echo -ne '\xde\xad\xc0\xde' >> /opt/openwrt/k1/build_dir/linux-ixp4xx_generic/root.jffs2-64k
dd if=/opt/openwrt/k1/build_dir/linux-ixp4xx_generic/root.jffs2-64k of=/opt/openwrt/k1/bin/ixp4xx/openwrt-ixp4xx-generic-jffs2-64k.img bs=65536 conv=sync 		# FIXME: removing this line will cause strange behaviour in the foreach loop below
461+1 records in
462+0 records out
30277632 bytes (30 MB) copied, 0.0493945 s, 613 MB/s
/opt/openwrt/k1/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -x zlib -x lzma -e 128KiB -o /opt/openwrt/k1/build_dir/linux-ixp4xx_generic/root.jffs2-128k -d /opt/openwrt/k1/build_dir/target-armeb_v5te_uClibc-0.9.30.3_eabi/root-ixp4xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 6816 blocks (23063669)  decompr: 0 blocks
      zlib (prio:80) - compr: 0 blocks (0/0)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 2472 blocks (6288770/8424025)  decompr: 0 blocks 
Compression errors: 0

comment:9 Changed 8 years ago by acoul

what is the output of:

grep LZMA build_dir/linux-ixp4xx_generic/linux-2.6.32.16/.config

you may try:

make oldconfig; make kernel_oldconfig; grep LZMA build_dir/linux-ixp4xx_generic/linux-2.6.32.16/.config

comment:10 Changed 8 years ago by Mark Kelly <mekelly5@…>

$ grep LZMA build_dir/linux-ixp4xx_generic/linux-2.6.32.16/.config
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_KERNEL_LZMA=y
CONFIG_RD_LZMA=y
CONFIG_SQUASHFS_LZMA=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_LZMA_NEEDED=y

comment:11 Changed 8 years ago by Mark Kelly <mekelly5@…>

off wait, that output above is not correct for whats in trunk version of the config, I have added a couple of those options from 2.6.33 config to the 2.6.32 to figure what's missing.

Index: config-2.6.32
===================================================================
--- config-2.6.32	(revision 22417)
+++ config-2.6.32	(working copy)
@@ -33,6 +33,8 @@
 CONFIG_CPU_TLB_V4WBI=y
 CONFIG_CPU_XSCALE=y
 # CONFIG_DEBUG_USER is not set
+CONFIG_JFFS2_LZMA=y
+CONFIG_LZMA_COMPRESS=y
 CONFIG_DECOMPRESS_LZMA=y
 CONFIG_DEVPORT=y
 # CONFIG_DM9000 is not set

but it's still not working

comment:12 Changed 8 years ago by anonymous

Here it is for me, if it helps:

$ grep LZMA build_dir/linux-ar71xx_generic/linux-2.6.32.16/.config
CONFIG_RD_LZMA=y
CONFIG_SQUASHFS_LZMA=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_LZMA_NEEDED=y
$ make oldconfig; make kernel_oldconfig; grep LZMA build_dir/linux-ar71xx_generic/linux-2.6.32.16/.config
CONFIG_RD_LZMA=y
CONFIG_SQUASHFS_LZMA=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_LZMA_NEEDED=y

comment:13 Changed 8 years ago by anonymous

Output for r22414-without-r22374:

trunk22414m22374/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -X zlib -x lzma -e 64KiB -o trunk22414m22374/build_dir/linux-ar71xx_generic/root.jffs2-64k -d trunk22414m22374/build_dir/target-mips_r2_uClibc-0.9.30.1/root-ar71xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 61 blocks (10696)  decompr: 0 blocks
      zlib (prio:80) + compr: 1380 blocks (2329516/4686395)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 1 blocks (38/39)  decompr: 0 blocks 
Compression errors: 0

trunk22414m22374/staging_dir/host/bin/mkfs.jffs2 --pad --big-endian --squash -v -X rtime -X zlib -x lzma -e 128KiB -o trunk22414m22374/build_dir/linux-ar71xx_generic/root.jffs2-128k -d trunk22414m22374/build_dir/target-mips_r2_uClibc-0.9.30.1/root-ar71xx -v 2>&1 1>/dev/null | awk '/^.+$/'
Compression mode: priority
Compressors:
      none             compr: 59 blocks (10488)  decompr: 0 blocks
      zlib (prio:80) + compr: 1366 blocks (2327277/4686603)  decompr: 0 blocks 
      lzma (prio:70) - compr: 0 blocks (0/0)  decompr: 0 blocks 
     rtime (prio:50) + compr: 1 blocks (38/39)  decompr: 0 blocks 
Compression errors: 0

Also, output of

grep LZMA build_dir/linux-ar71xx_generic/linux-2.6.32.16/.config

is the same as I posted above, both before and after the

make oldconfig; make kernel_oldconfig

. Is that expected?

comment:14 Changed 8 years ago by acoul

how about:

grep JFFS2_ZLIB build_dir/linux-ar71xx_generic/linux-2.6.32.16/.config

try to resync your image.mk

rm include/image.mk; svn up

and perhaps:

rm target/linux/ar71xx/config-2.6.32; svn up

comment:15 Changed 8 years ago by acoul

make oldconfig; make kernel_oldconfig

is a good practice after updating your trunk

rm -rf build_dir/ staging_dir/ tmp/*

also helps to clean old stale files and it's not as destructive as:

make distclean

comment:16 Changed 8 years ago by acoul

  • Resolution set to fixed
  • Status changed from reopened to closed

this issue should be finally fixed in r22423. thank you for reporting.

comment:17 Changed 4 years ago by jow

  • Milestone changed from Attitude Adjustment 12.09 to Barrier Breaker 14.07

Milestone Attitude Adjustment 12.09 deleted

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.