Modify

Opened 4 years ago

Closed 3 years ago

Last modified 2 years ago

#14076 closed defect (wontfix)

layer7 netfilter module is not created in recent trunk

Reported by: duvi Owned by: developers
Priority: normal Milestone: Chaos Calmer 15.05
Component: kernel Version: Trunk
Keywords: Cc:

Description

kmod-ipt-filter and iptables-mod-filter are selected, but xt_layer7.ko is not created on ar71xx, and probably on everything since the switch to kernel 3.10

Attachments (5)

632-netfilter_layer7_3.10.x_experimental_fix.patch (327 bytes) - added by n3ph 4 years ago.
632-netfilter_layer7_3.10.x_experimental_fix.patch
633-netfilter_layer7_3.10.x_procfs_fix.patch (1.7 KB) - added by n3ph 4 years ago.
633-netfilter_layer7_3.10.x_procfs_fix.patch
633-netfilter_layer7_3.10.x_procfs_fix.2.patch (1.6 KB) - added by duvi 4 years ago.
604-netfilter_layer7_experimental_fix.patch (377 bytes) - added by ben@… 4 years ago.
Patch to 603-netfilter_layer7_2.6.36_fix.patch
xt_layer7.c (17.0 KB) - added by bobby846@… 2 years ago.
layer7 porting to kernel 3.4

Download all attachments as: .zip

Change History (54)

comment:1 Changed 4 years ago by anonymous

You need to select iptables-mod-ipp2p to use Layer 7 filtering.
It will pull in kmod dependencies as well.

comment:2 Changed 4 years ago by duvi

I'm sorry but I don't see what ipp2p has to do with layer7.

  1. Before kernel 3.10 I never compiled iptables-mod-ipp2p and still had layer7 filter.
  2. Taken from 3.10 make menuconfig
    CONFIG_PACKAGE_iptables-mod-filter:
    iptables extensions for packet content inspection.
    Includes support for:
    Matches:
     - layer7
     - string
    
  3. Just to prove, I selected iptables-mod-ipp2p and rebuilt the whole system. layer7 filter was still neither compiled into iptables-mod-filter nor into iptables-mod-filter (and the relevant kernel modules).

comment:3 Changed 4 years ago by anonymous

I can confirm this bug is present in kernel 3.10.10 under same conditions

comment:4 follow-up: Changed 4 years ago by christian@…

it's due to

depends on EXPERIMENTAL

definition in its corresponding Kconfig.. CONFIG_EXPERIMENTAL is not existent in Kernel 3.10+
that's why the module is not build at all using the patches applied..

but if you remove the dependency in Kconfig and are able to select the option again, it breaks compiling as of https://lwn.net/Articles/549737/ (killed API create_proc_entry() )

comment:5 in reply to: ↑ 4 Changed 4 years ago by christian@…

Replying to christian@…:
patch is here (sorry for the inline - I have not managed to use the attachment function in trac ;) ):

# patches-3.10/604-netfilter_layer7_3.10.x_fix.patch
diff -Naur linux-3.10.7/net/netfilter/Kconfig linux-3.10.7.patched/net/netfilter/Kconfig
--- linux-3.10.7/net/netfilter/Kconfig  2013-09-18 23:52:59.000000000 +0200
+++ linux-3.10.7.patched/net/netfilter/Kconfig  2013-09-18 20:13:51.000000000 +0200
@@ -1013,7 +1013,6 @@
 
 config NETFILTER_XT_MATCH_LAYER7
        tristate '"layer7" match support'
-       depends on EXPERIMENTAL
        depends on NETFILTER_XTABLES
        depends on NETFILTER_ADVANCED
        depends on NF_CONNTRACK
diff -Naur linux-3.10.7/net/netfilter/xt_layer7.c linux-3.10.7.patched/net/netfilter/xt_layer7.c
--- linux-3.10.7/net/netfilter/xt_layer7.c      2013-09-18 23:52:00.000000000 +0200
+++ linux-3.10.7.patched/net/netfilter/xt_layer7.c      2013-09-18 23:27:26.000000000 +0200
@@ -364,8 +364,7 @@
 }
 
 /* write out num_packets to userland. */
-static int layer7_read_proc(char* page, char ** start, off_t off, int count,
-                            int* eof, void * data)
+static ssize_t layer7_read_proc(struct file *filp, char __user *page, size_t count, loff_t *data)
 {
        if(num_packets > 99 && net_ratelimit())
                printk(KERN_ERR "layer7: NOT REACHED. num_packets too big\n");
@@ -375,14 +374,11 @@
        page[2] = '\n';
        page[3] = '\0';
 
-       *eof=1;
-
        return 3;
 }
 
 /* Read in num_packets from userland */
-static int layer7_write_proc(struct file* file, const char* buffer,
-                             unsigned long count, void *data)
+static ssize_t layer7_write_proc(struct file *file, const char __user *buffer, size_t count, loff_t *data)
 {
        char * foo = kmalloc(count, GFP_ATOMIC);
 
@@ -661,12 +657,20 @@
 }
 
 /* register the proc file */
-static void layer7_init_proc(void)
+static const struct file_operations layer7_fops = {
+       .owner  = THIS_MODULE,
+       .read   = layer7_read_proc,
+       .write  = layer7_write_proc,
+};
+
+static int layer7_init_proc(void)
 {
        struct proc_dir_entry* entry;
-       entry = create_proc_entry("layer7_numpackets", 0644, init_net.proc_net);
-       entry->read_proc = layer7_read_proc;
-       entry->write_proc = layer7_write_proc;
+       entry = proc_create("layer7_numpackets", 0644, init_net.proc_net, &layer7_fops);
+       if (entry == NULL) {
+               return -ENOMEM;
+       }
+       return 0;
 }
 
 static int __init xt_layer7_init(void)

comment:6 Changed 4 years ago by christian@…

this patch needs a profound REVIEW - it freezes the machine when initializing..
sorry for that.. but someone more skillful witch C and Kernel insight will hopefully get the idea..

comment:7 Changed 4 years ago by sicarrots

I can confirm lack of layer7 filters in current trunk. Is there any possibility to fix it?

comment:8 Changed 4 years ago by n3ph

Tried this patch with r38610 today and it seems to work .. No freeze on load - and /etc/init.d/freifunk-p2pblock doesn't blame about "iptables v1.4.20: Couldn't load match `layer7':File not found" anymore...

So ... Bump?

Changed 4 years ago by n3ph

632-netfilter_layer7_3.10.x_experimental_fix.patch

Changed 4 years ago by n3ph

633-netfilter_layer7_3.10.x_procfs_fix.patch

comment:9 Changed 4 years ago by n3ph

I also redone the patches with quilt and cutted them into two pieces.. So you can just copy them into target/linux/ar71xx/patches-3.10 to test them again...

comment:10 Changed 4 years ago by duvi

It doesn't seem to work for me.

Applying /home/duvi/openwrt/trunk/target/linux/generic/patches-3.10/632-netfilter_layer7_3.10.x_experimental_fix.patch using plaintext:
patching file net/netfilter/Kconfig
Hunk #1 FAILED at 1011.
1 out of 1 hunk FAILED -- saving rejects to file net/netfilter/Kconfig.rej
Patch failed!  Please fix /home/duvi/openwrt/trunk/target/linux/generic/patches-3.10/632-netfilter_layer7_3.10.x_experimental_fix.patch!

Although I've put the patches to target/linux/generic rather than target/linux/ar71xx since they shouldn't be device specific.

comment:11 Changed 4 years ago by duvi

Getting rid of "depends on EXPERIMENTAL" manually it also hangs on the second patch.

Applying /home/duvi/openwrt/trunk/target/linux/generic/patches-3.10/633-netfilter_layer7_3.10.x_procfs_fix.patch using plaintext:
patching file net/netfilter/xt_layer7.c
/usr/bin/patch: **** malformed patch at line 10: count, loff_

Patch failed!  Please fix /home/duvi/openwrt/trunk/target/linux/generic/patches-3.10/633-netfilter_layer7_3.10.x_procfs_fix.patch!

comment:12 follow-up: Changed 4 years ago by duvi

Okay, here's the steps to make it work.

  1. Delete line 9 "+ depends on EXPERIMENTAL" from target/linux/generic/patches-3.10/603-netfilter_layer7_2.6.36_fix.patch
  1. Upload my attached 633-netfilter_layer7_3.10.x_procfs_fix.patch to target/linux/generic

Thank you christian and n3ph!

WARNING: the patch has only been compile tested, so xt_layer7.ko is created, but I've not tested it on a device yet, so it might still freeze the machine as christian said in comment 6.

comment:13 Changed 4 years ago by duvi

Tested on ar71xx Mikrotik RB433UAH, and it seems to work, no freeze seen.

comment:14 Changed 4 years ago by n3ph

Confirming on RB951G-2HnD

  • r38663
  • modified 603-netfilter_layer7_2.6.36_fix.patch (Remove "depends on EXPERIMENTAL")
  • 633-netfilter_layer7_3.10.x_procfs_fix.2.patch

No freeze - but i have no real traffic ongoing.. so there are no matching packets for layer7-filtering anyhow.. Next days in test-env it will show up ...

And right - its not hardware-specific. so target/linux/generic should be fine

comment:15 Changed 4 years ago by christian@…

ok - for completeness: I'm on (Gentoo-)x86 @ Atom 230.. ;)

my freeze above might also originate from that I've patched module w/i the build-tree of the currently running Kernel while patching and (re)building only the module. As soon as it had built successfully, I insmoded it.. ;)

Alas, that is the only machine I've to test if it's actually working - and that's my productive main router..

comment:16 Changed 4 years ago by jerzmid

Tested on PLD with kernel kernel-10.03.20. Applying patch 633-netfilter_layer7_3.10.x_procfs_fix.2.patch freezes system. Does anyone know the solution to the problem

comment:17 Changed 4 years ago by anonymous

Please fix

comment:18 Changed 4 years ago by rav1ne

Argh...so that's why my firewall is dead. Fortunately Google found this topic, but as i see, no instant solution in the trunk. *feeling sad*

comment:19 Changed 4 years ago by anonymous

Why isn't the patch in trunk?

comment:20 Changed 4 years ago by anonymous

i can confirm this bug report is still valid as of r39185 on ar71xx

xt_layer7.ko is not created and every layer7 entry in qos config result in this error on system log:

Thu Jan  2 21:24:08 2014 user.emerg syslog: iptables v1.4.20: Couldn't load match `layer7':No such file or directory
Thu Jan  2 21:24:08 2014 user.emerg syslog: 
Thu Jan  2 21:24:08 2014 user.emerg syslog: Try `iptables -h' or 'iptables --help' for more information.

please fix.
tia

comment:21 Changed 4 years ago by anonymous

I can confirm this is still an issue (I am using the Trunk image builder). Unfortunately I'm not smart enough to work out how to manually compile the firmware from scratch.

How do these patches find there way in to the trunk builds and therefore in to the Image Builder builds?

M

comment:22 Changed 4 years ago by anonymous

*bump* - Please merge the patch upstream....

comment:23 in reply to: ↑ 12 Changed 4 years ago by AdamK

Replying to duvi:

Okay, here's the steps to make it work.

  1. Delete line 9 "+ depends on EXPERIMENTAL" from target/linux/generic/patches-3.10/603-netfilter_layer7_2.6.36_fix.patch

In the current trunk source this now breaks the patch file giving an error as follows:

Applying /home/adamk/trunk/target/linux/generic/patches-3.10/603-netfilter_layer7_2.6.36_fix.patch using plaintext:
patching file net/netfilter/Kconfig
/usr/bin/patch: malformed patch at line 30: @@ -1203,26 +1224,11 @@ config NETFILTER_XT_MATCH_STATE

Patch failed! Please fix /home/adamk/trunk/target/linux/generic/patches-3.10/603-netfilter_layer7_2.6.36_fix.patch!

  1. Upload my attached 633-netfilter_layer7_3.10.x_procfs_fix.patch to target/linux/generic

This patch seems to run successfully however layer7 is still not in the built image.

Am testing on a TP-Link WDR-4300...keen to see layer7 stuffs working in Barrier Breaker! Build server is Ubuntu 12.04.4

comment:24 Changed 4 years ago by duvi

The patches are still working for me on latest trunk.

comment:25 Changed 4 years ago by anonymous

*bump* please get this upstream

comment:26 Changed 4 years ago by ben@…

For me, compiling trunk r40746 for atheros target fails on the manually edited patch, just as AdamK describes in comment # 23 above.

I'm including my own patch modify 603-netfilter_layer7_2.6.36_fix.patch, which seems to permit successful compilation. Haven't yet had the chance to try functional testing of layer7 module.

So, confirming successful compile of trunk r40746 for atheros target with the following patches added to target/linux/generic/patches-3.10:

633-netfilter_layer7_3.10.x_procfs_fix.2.patch​ from comment # 12
604-netfilter_layer7_experimental_fix.patch attached below

Changed 4 years ago by ben@…

Patch to 603-netfilter_layer7_2.6.36_fix.patch

comment:27 Changed 4 years ago by anonymous

:-(

Whats up here? Is someone actually maintaining this stuff?

comment:28 Changed 4 years ago by anonymous

Should be marked as a blocker for the upcoming barrier breaker release

comment:29 Changed 4 years ago by anonymous

+1 - definitely

comment:30 Changed 3 years ago by Diffie

I wanted to say that the patch in this ticket applies cleanly (with some offset) to 3.10.54 kernel, compiles and runs fine on ar71xx platform (netgear wndr3700v4). The system log does not show any more layer7 errors.

This has been compile and run tested, hope this will go into BB final.

comment:31 Changed 3 years ago by pharaoh@…

seems BB goes final without layer7. just checked the the 14.07 image and packages. QoS generates files wth layer7 statements but there is no layer 7 module:

root@wlanrouter:/usr/lib/qos# cat generate.sh | grep layer

*:layer7)

add_insmod ipt_layer7
add_insmod xt_layer7
append "$var" "-m layer7 --l7proto $value${pkt:+ --l7pkt}"

opkg files iptables-mod-filter
Package iptables-mod-filter (1.4.21-1) is installed on root and has the following files:
/etc/l7-protocols/msnmessenger.pat
/etc/l7-protocols/jabber.pat
/etc/l7-protocols/vnc.pat
/etc/l7-protocols/ntp.pat
/etc/l7-protocols/http.pat
/usr/lib/iptables/libxt_string.so
/etc/l7-protocols/irc.pat
/etc/l7-protocols/smtp.pat
/etc/l7-protocols/bittorrent.pat
/etc/l7-protocols/gnutella.pat
/etc/l7-protocols/ssl.pat
/etc/l7-protocols/ident.pat
/etc/l7-protocols/aim.pat
/etc/l7-protocols/ftp.pat
/etc/l7-protocols/edonkey.pat
/etc/l7-protocols/fasttrack.pat
/etc/l7-protocols/pop3.pat

no xt_layer7 in 14.07 ;-(

comment:32 Changed 3 years ago by anonymous

I can confirm that neither BB or RC3 xt_layer7.ko trunk module is created, I am using brcm47xx / mips74k (WRT320N), so the qos rules fails at applying layer7 statements

comment:33 Changed 3 years ago by anonymous

I can confirm that neither BB or RC3 xt_layer7.ko trunk module is created, so the qos rules fails at applying layer7 statements, I am using brcm47xx / mips74k (WRT320N)

comment:34 Changed 3 years ago by arokh

Using layer 7 filtering causes kernel crashes. Looks like the code hasn't been worked on in years and might be abandoned. Probably best not to include it until someone revises the code.

comment:35 Changed 3 years ago by kl522@…

Tried all the patches and mods on kernel 3.10.x.

Basically the xt_layer7.ko is created and can be loaded. But it could not do any matching, basically the patch 601-netfilter_layer7_pktmatch.patch is not applicable for kernel 3.10.x anymore. I inserted some printk at the match function, basically the flow is different from a worker older kernel, it could not go down to match the traffic at all.

comment:36 Changed 3 years ago by kl522@…

I mean "working" older kernel ....

comment:37 Changed 3 years ago by ben@…

From observation that kernel v3.10 layer7 patches presently in Barrier Breaker and trunk are no longer functional, some googling yielded what appears to be active layer7 integration into the v3.10 and now v3.14 kernel by the IPFire group:
http://lists.ipfire.org/pipermail/ipfire-scm/2014-April/003185.html
https://github.com/ipfire/ipfire-2.x/blob/master/src/patches/linux-3.14-layer7-filter.patch
https://github.com/ipfire/ipfire-2.x/blob/bedebe884765e82184ed1bcc0a5a551b4d479b03/src/patches/linux-3.10-layer7-filter.patch

Anyone perhaps aware if IPFire has functional layer 7 matching?

comment:38 Changed 3 years ago by kl522@…

Tested the linux-3.10-layer7-filter.patch. Did not test the 3.14 version.

Last commit was 10 months ago.

No, it is not functional. The match is not working.

comment:39 Changed 3 years ago by kl522@…

The IPfire also has a iso image. The layer7 matching is working on the isoimage. So I am checking why my testing based on the source patch does not seem to work while their kernel 3.10.44 isoimage seem to work.

comment:40 Changed 3 years ago by duvi

With the following patches (603: modified, 633: added) L7 filtering works for me in trunk with kernel 3.14 on ar71xx.
Place them into target/linux/generic/patches-3.14

comment:42 Changed 3 years ago by mctiew@…

Sorry got a bit confused with the last two posts.

Is it working or is it not working ? :)

comment:43 Changed 3 years ago by duvi

L7 filter works, adding files doesn't.

comment:44 Changed 3 years ago by nbd

  • Resolution set to wontfix
  • Status changed from new to closed

layer7 has been removed, see r45423

comment:45 Changed 3 years ago by duvi

Is (was) working fine for me. Noone was forced to use it, I don't agree with this decision.

comment:46 Changed 3 years ago by nbd

It looks like layer7 was responsible for a recent round of incomprehensible kernel crash tickets that were submitted by a bunch of people trying to use it.
It also increases the size of the netfilter per-connection data for everybody.
Its code is a big pile of garbage that I never ever want to waste any precious debugging time on again. It has caused me enough trouble over the years already.

If somebody finds a way to rework the code to build as a standalone package without bloating up core data structures or causing nasty crashes, I'll happily take it back. But until that time, it shall remain out of tree, and I will ignore any weird looking crashes of devices that have it loaded.

comment:47 Changed 3 years ago by ben@…

A possible replacement for layer7 matching could be using the iptables string match module instead. For example, I've used this to match bittorrent traffic:

iptables -I p2pblock -m string --string "BitTorrent protocol" --algo bm -m recent --rdest --set --name P2PBLOCK
iptables -I p2pblock -m string --string "BitTorrent protocol" --algo bm -m limit --limit 1/minute -j LOG --log-prefix P2PBLOCK-seen-bitbm:

You could try this approach, porting the strings/patterns from the old layer7 code.

comment:48 Changed 3 years ago by duvi

Understood, thanks for explaining.

Thanks for the tip, I'll try that.

Changed 2 years ago by bobby846@…

layer7 porting to kernel 3.4

comment:49 Changed 2 years ago by bobby846@…

i am trying to port l7filter on kernel 3.4.i did the changes layer 7 accordingly and i am able to apply the rules ./iptables -I OUTPUT -m layer7 --l7proto ssh -j DROP,(and also for INPUT) where i can see them applied using ./iptables -L.but when i try to ssh,it is not blocking the protocol,it is allowing me ssh.i started debugging the xt_layer7.c (inserting xt_layer7.ko in to kernel).and found that i am unable to get master connection tracking ---- while (master_ct(master_conntrack) != NULL) at line 463.so it is ubale to compare with the protocol pattern.can some one give me ideas how to get the master connection tarcking getting worked..

i have attached xt_layer7.c

Thanks and regards
bobby

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.