Modify

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#14189 closed defect (fixed)

WDR-4300 - IO charset iso8859-1 not found

Reported by: katananja@… Owned by: developers
Priority: normal Milestone: Barrier Breaker 14.07
Component: kernel Version: Trunk
Keywords: Cc:

Description

WDR-4300 v1.6 trunk r38055

Compile with USB support and it doesn't work when I try to mount it.

Attachments (2)

wdr4300.config.tar.gz (12.7 KB) - added by katananja@… 4 years ago.
Kernel config file
iso8859.log (14.1 KB) - added by joykicer@… 4 years ago.
I have the same issue when built for a 6328 device.

Download all attachments as: .zip

Change History (22)

Changed 4 years ago by katananja@…

Kernel config file

comment:1 follow-ups: Changed 4 years ago by jow

Is it actually loaded? Check

lsmod | iso8859

comment:2 in reply to: ↑ 1 Changed 4 years ago by joykicer@…

Replying to jow:

Is it actually loaded? Check

lsmod | iso8859

No, not loaded. Attachment is the dmesg log.

May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: missing module (null)
May 21 08:10:01 sysinit: kmod: ran 48 iterations
May 21 08:10:01 sysinit: kmod: 2 modules could not be probed
May 21 08:10:01 sysinit: kmod: - bcma - 0
May 21 08:10:01 sysinit: kmod: - nls_iso8859_1 - 0

Changed 4 years ago by joykicer@…

I have the same issue when built for a 6328 device.

comment:3 in reply to: ↑ 1 Changed 4 years ago by katananja@…

Replying to jow:

Is it actually loaded? Check

lsmod | iso8859

In fact it's not loading.

find / -name *iso8859*
/etc/modules.d/25-nls-iso8859-1
/lib/modules/3.10.12/nls_iso8859-1.ko
/rom/etc/modules.d/25-nls-iso8859-1
/rom/lib/modules/3.10.12/nls_iso8859-1.ko
/rom/usr/lib/opkg/info/kmod-nls-iso8859-1.control
/rom/usr/lib/opkg/info/kmod-nls-iso8859-1.list
find: /sys/kernel/slab/L2TP/IP: No such file or directory
/usr/lib/opkg/info/kmod-nls-iso8859-1.control
/usr/lib/opkg/info/kmod-nls-iso8859-1.list
insmod nls_iso8859
kmod: missing module (null)
kmod: failed to insert (null)

I've set this module to be build inside the kernel.

comment:4 Changed 4 years ago by gonzzo

I have the same problem with trunk update r38114 installed on a TP-Link TL-WR842N/ND v1.

  • same missing module (null) at sysinit
  • moude iso8859-1 not loaded
  • can't mount vfat usb hdd

comment:5 Changed 4 years ago by hnyman <hannu.nyman@…>

Currently (since r38019) insmod is a symlink to "kmodloader" in ubox. Looks like the new ubox/procd kmodloader does not handle properly either dash & underscore in the modules name.

Manually loading the nls-iso8859-1 module with the old busybox insmod works and enables me to mount drives using the iso8859-1 codepage:

/sbin/insmod nls_iso8859-1

Most likely bug #14182 filed by me is related to the same underlying issue.

comment:6 Changed 4 years ago by hnyman <hannu.nyman@…>

As far as I am able to debug, loading a module with both "_" and "-" in its name (like nls_iso8859-1.ko) will always fail.

I think that the problem lies at "get_module_path" function of kmodloader in ubox.
http://nbd.name/gitweb.cgi?p=luci2/ubox.git;a=blob;f=kmodloader.c;h=ab8a00bc9f089498042a01bde06856c335009568;hb=102b1ad4c5c4db2c5a62f8581ab098885d4295c8#l102

The function tries to find the full path of the module to be loaded.

First it tries if full path was given as the argument.
Then it tries the default modulepath with the modulename all "-" replaced by "_"
Then it tries the default modulepath with the modulename all "_" replaced by "-"

But a module with both "_" and "-" in its name never gets properly tried...

comment:8 Changed 4 years ago by hnyman <hannu.nyman@…>

That patch does not help. I already tested it :-(

It only prevents _/- replacement in the path, but does not affect the replacement in the actual module name.

I wrote my own patch and am currently compiling it. (default modulepath with unchanged module name before trying the two replaces). I will report if it works.

comment:9 Changed 4 years ago by matus

I was able to find a work-around for this issue by creating a hard link to the module using only '_' and rebooting the system

ln /lib/modules/3.10.12/nls_iso8859-1 /lib/modules/3.10.12/nls_iso8859_1
reboot

after this, I could mount the VFat partition without a problem

hope it helps until a more definitive solution is developed

comment:10 follow-up: Changed 4 years ago by katananja@…

The problem it's that this is not the "only" module, there is a lot of then with the same exact error message.

comment:11 Changed 4 years ago by matus

clearly, this is by no means a permanent solution, but a work-around to be able to use the needed modules in the meanwhile.

one could develop a shell script to make the links automatically and run it after installing a conflictive module

again, just ideas for the time being

comment:12 in reply to: ↑ 10 Changed 4 years ago by hnyman

Replying to katananja@…:

The problem it's that this is not the "only" module, there is a lot of then with the same exact error message.

I think that the "missing module" is a different issue. It is probably the dependency search which apparently produces lots of error messages. I am not sure if those are actual errors or merely too active lookup messages.

This 8859-1 error is about having both _ and - in the module file name. But there are more such modules, so this needs to be fixed asap.

I have tried to search for solutions and interestingly google reveals old reports about similar errors with original modprobe years ago, so this is not unique to Openwrt implementation of the modprobe.
https://www.google.com/search?q=modprobe+8859+error

I will try tonight to reverse r38021 regarding nls-base, and will revert it to the old-fashioned autoload. the codepages might be loaded via the nls-base, so that might help.
https://dev.openwrt.org/changeset/38021#file11

comment:13 Changed 4 years ago by hnyman <hannu.nyman@…>

That didn't help.

After some more debugging, it is even more complicated:

Looks like the function get_module_name already mangles (= tries to clean) the module name by replacing all dashes with underscore, thus the module name "nls_8859-1" gets simplified to "nls_8859_1".
http://nbd.name/gitweb.cgi?p=luci2/ubox.git;a=blob;f=kmodloader.c;hb=102b1ad4c5c4db2c5a62f8581ab098885d4295c8#l127

When that is then later used in get_module_path, both "all _" and "all -" are tested, but as the real module file has both of them, it is never found. My previous attempt on Monday to use the original module name failed, because the module name was not the original one, but mangled by get_module_name :-(

Added logging shows that clearly:

root@OpenWrt:~# insmod nls_8859-1
kmod: get_module_name: nls_8859-1 --> nls_8859_1
kmod: get_module_path: nls_8859_1
kmod: get_module_path_dash: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path_uscore: /lib/modules/3.10.12/nls-8859-1.ko
kmod: get_module_path_intact: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path not found: nls_8859_1
kmod: missing module (null)
kmod: get_module_path: nls_8859_1
kmod: get_module_path_dash: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path_uscore: /lib/modules/3.10.12/nls-8859-1.ko
kmod: get_module_path_intact: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path not found: nls_8859_1
kmod: get_module_path: nls_8859_1
kmod: get_module_path_dash: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path_uscore: /lib/modules/3.10.12/nls-8859-1.ko
kmod: get_module_path_intact: /lib/modules/3.10.12/nls_8859_1.ko
kmod: get_module_path not found: nls_8859_1
kmod: failed to insert (null)

The real path /lib/modules/3.10.12/nls_8859-1.ko is never tested.

I will next try removing the "replace - with _" logic from get_module_name and hope that it does not break anything.

comment:14 Changed 4 years ago by hnyman <hannu.nyman@…>

The previous log example had wrong filename, but that did not actually decrease the value of the path parsing logic example.

I compiled a version, where get_module_name does not mangle the module name. That enables insmod to work again:

root@OpenWrt:~# lsmod | grep 8859
root@OpenWrt:~# insmod nls_iso8859-1
kmod: get_module_name: nls_iso8859-1 --> nls_iso8859-1
kmod: get_module_path: nls_iso8859-1
kmod: get_module_path_intact: /lib/modules/3.10.12/nls_iso8859-1.ko
root@OpenWrt:~# lsmod | grep 8859
nls_base                4974 10 nls_iso8859_1
nls_iso8859_1           2880  0

The module did not get loaded automatically at the reboot or when inserting the USB stick, so this is not the full solution, yet.

comment:15 Changed 4 years ago by hnyman <hannu.nyman@…>

The previous solution of removing dash replacement from get_module_name caused three other modules not to load: leds-wndr3700-usb, ledtrig-netdev, ledtrig-usbdev

Apparently that replacement is still needed for those modules. so removing that is not a good solution.

comment:16 Changed 4 years ago by hnyman <hannu.nyman@…>

I created a patch for ubox that works for me. And the nls_iso8859-1 module gets now properly loaded at the startup.

The original reason lies with Linux sources. Some of the nls code page module file name are strange, as they contain both _ and -. Link to Linux sources: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/fs/nls

Modprobe implementations apparently try to convert most module names to underscores, but finding those mixed-name nls files will then be impossible.

There are 4 basic ways to solve:
1) Change filenames in Linux sources (and in module definitions). Or duplicate the files (nls_iso8859_1.ko). Or something like that.
2) Abandon ubox modprobe & insmod. Busybox insmod works ok. Not likely, as Openwrt has just introduced ubox based modprobe/insmod.
3) Handle nls codepage module names differently in modprobe = ubox. I tried that earlier this week, but apparently the modprobe uses various functions to handle the module name, so trying to preserve original mixed name for nls* did not quite succeed.
4) Handle file search for the nls module files differently in modprobe = ubox. If the module name starts with "nls", assume that it may contain later a dash "-", despite it being already standardized to underscore "_" in the module name. Note that there are no nls file names with a second _ after the original nls_ .

I created an ugly hack/patch that works for me. I applied it to ubox head, one check-in ahead of the ubox currently brought to Openwrt, but it should be applicable also to the current ubox version.

This requires creating a new patches directory for ubox in the Openwrt buildroot and creating the patch there.

After failing the normal file search, the patch checks if the module name starts with nls, and if yes, then it jumps over the initial "nls_" but replaces the later occurences of _ with -, making the filename once again like "nls_iso8859-1.ko".

The new check has been placed last, just before the failure and return with NULL. So, all normal modules are first handled with the normal logic and this is just a fallback for nls*. (I have not checked if there are other modules with mixed names.)

--- trunk/package/system/ubox/Makefile	(revision 38237)
+++ trunk/package/system/ubox/Makefile	(working copy)
@@ -1,13 +1,13 @@
 include $(TOPDIR)/rules.mk
 
 PKG_NAME:=ubox
-PKG_VERSION:=2013-09-17
+PKG_VERSION:=2013-09-27
 PKG_RELEASE=$(PKG_SOURCE_VERSION)
 
 PKG_SOURCE_PROTO:=git
 PKG_SOURCE_URL:=git://nbd.name/luci2/ubox.git
 PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
-PKG_SOURCE_VERSION:=cc07b0b7ac66d0e4a05470dd69a28305405cd7dd
+PKG_SOURCE_VERSION:=102b1ad4c5c4db2c5a62f8581ab098885d4295c8
 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-$(PKG_SOURCE_VERSION).tar.gz
 CMAKE_INSTALL:=1
 
--- trunk/package/system/ubox/patches/020-add-logging	(revision 0)
+++ trunk/package/system/ubox/patches/020-add-logging	(working copy)
@@ -0,0 +1,26 @@
+--- a/kmodloader.c
++++ b/kmodloader.c
+@@ -121,6 +121,23 @@
+ 	if (!stat(path, &s))
+ 		return path;
+ 
++	// Fix for nls module names like nls_iso8859-1.ko, revert the last underscore
++	char *t;
++	if (!strncmp("nls", name, 3)) {
++		snprintf(path, 256, "%s" DEF_MOD_PATH "%s.ko", prefix, ver.release, name);
++		t = basename(path);		
++		replace_dash(t);
++		t += 4;   // jump over "nls-" at the start
++        	while (t && *t) {
++			if (*t == '_')
++				*t = '-';
++			t++;
++		}
++		//LOG("get_module_path: %s\n", path);
++		if (!stat(path, &s))
++			return path;
++	}
++
+ 	return NULL;
+ }
+ 

#14182 is a duplicate of this bug.

comment:17 Changed 4 years ago by jow

Unfortunately the nls_* modules are not the only exception. Doing the following oneliner on a Debian 7 machine with a 3.10 backport kernel reveals the following module names:

$ find /lib/modules/3.10-0.bpo.2-686-pae/ -name '*.ko' | xargs -L1 basename | sort -u | grep -E '(.+-.+_.+|.+_.+-.+)'
ad525x_dpot-i2c.ko
ad525x_dpot-spi.ko
ir-mce_kbd-decoder.ko
nls_euc-jp.ko
nls_iso8859-13.ko
nls_iso8859-14.ko
nls_iso8859-15.ko
nls_iso8859-1.ko
nls_iso8859-2.ko
nls_iso8859-3.ko
nls_iso8859-4.ko
nls_iso8859-5.ko
nls_iso8859-6.ko
nls_iso8859-7.ko
nls_iso8859-9.ko
nls_koi8-r.ko
nls_koi8-ru.ko
nls_koi8-u.ko
old_belkin-sir.ko

So a more generic solution is needed still.

comment:18 Changed 4 years ago by jow

  • Resolution set to fixed
  • Status changed from new to closed

comment:19 Changed 4 years ago by anonymous

Thanks, fixed also for DIR-505 A1

comment:20 Changed 4 years ago by jow

  • Milestone changed from Attitude Adjustment 12.09 to Barrier Breaker 14.07

Milestone Attitude Adjustment 12.09 deleted

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.