From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Core Update 186 (testing) report Date: Tue, 28 May 2024 11:05:08 +0100 Message-ID: <01B1F4A4-F8CE-43E8-82A1-AD2C95DFD3A0@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============4592867150562769305==" List-Id: --===============4592867150562769305== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, Just to update this thread I can only say that there must be some minor bug h= ere. Searching through the recent commits in the kernel Git repository, there= have not been any changes that I would obviously connect with this. So the m= ost likely explanation is a rare race condition that might have been newly in= troduced or long existing. We don=E2=80=99t know. That Suricata then starts running in a loop eating all the memory until it is= finally killed is obviously a bad thing. So I suggest you report both proble= ms to the respective upstreams and we see what we hear back from them. I don= =E2=80=99t think that this should be a release blocker. Best, -Michael > On 19 May 2024, at 16:37, Peter M=C3=BCller wr= ote: >=20 > Hello *, >=20 > I'm afraid I spoke too soon: Today, for unknown reasons, Suricata triggered= the OOM killer: >=20 > May 19 11:49:26 maverick kernel: Suricata-Main invoked oom-killer: gfp_mask= =3D0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=3D0, oom_score_adj=3D0 > May 19 11:49:26 maverick kernel: CPU: 3 PID: 5196 Comm: Suricata-Main Taint= ed: G D 6.6.30-ipfire #1 > May 19 11:49:26 maverick kernel: [ 5196] 101 5196 280087 115466 1= 634304 72864 0 Suricata-Main > May 19 11:49:26 maverick kernel: oom-kill:constraint=3DCONSTRAINT_NONE,node= mask=3D(null),cpuset=3D/,mems_allowed=3D0,global_oom,task_memcg=3D/,task=3DSu= ricata-Main,pid=3D5196,uid=3D101 > May 19 11:49:26 maverick kernel: Out of memory: Killed process 5196 (Surica= ta-Main) total-vm:1120348kB, anon-rss:461608kB, file-rss:256kB, shmem-rss:0kB= , UID:101 pgtables:1596kB oom_score_adj:0 >=20 > Attached to this e-mail is the memory consumption graph, which corroborates= that, starting > at around 10:40 AM, something was eating up an extraordinary amount of memo= ry. The following > might be related: >=20 > May 19 10:41:22 maverick kernel: BUG: kernel NULL pointer dereference, addr= ess: 0000000000000000 > May 19 10:41:22 maverick kernel: #PF: supervisor instruction fetch in kerne= l mode > May 19 10:41:22 maverick kernel: #PF: error_code(0x0010) - not-present page > May 19 10:41:22 maverick kernel: PGD 0 P4D 0=20 > May 19 10:41:22 maverick kernel: Oops: 0010 [#1] PREEMPT SMP PTI > May 19 10:41:22 maverick kernel: CPU: 0 PID: 1585 Comm: tor Not tainted 6.6= .30-ipfire #1 > May 19 10:41:22 maverick kernel: Hardware name: > May 19 10:41:22 maverick kernel: RIP: 0010:0x0 > May 19 10:41:22 maverick kernel: Code: Unable to access opcode bytes at 0xf= fffffffffffffd6. > May 19 10:41:22 maverick kernel: RSP: 0018:ffffc90000433900 EFLAGS: 00010246 > May 19 10:41:22 maverick kernel: RAX: 0000000000000000 RBX: ffff88814cd371c= 0 RCX: 0000000000000000 > May 19 10:41:22 maverick kernel: RDX: 0000000000000000 RSI: ffffc900004339d= 8 RDI: 0000000000000000 > May 19 10:41:22 maverick kernel: RBP: 0000000000000218 R08: 000000000000000= 0 R09: 0000000000000000 > May 19 10:41:22 maverick kernel: R10: 0000000000000000 R11: 000000000000000= 0 R12: 0000000000000218 > May 19 10:41:22 maverick kernel: R13: ffff888101cf2d00 R14: 000000000000004= 0 R15: ffffc900004339d8 > May 19 10:41:22 maverick kernel: FS: 0000722d08168740(0000) GS:ffff88817bc= 00000(0000) knlGS:0000000000000000 > May 19 10:41:22 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008= 0050033 > May 19 10:41:22 maverick kernel: CR2: ffffffffffffffd6 CR3: 000000010a9ee00= 0 CR4: 00000000001006f0 > May 19 10:41:22 maverick kernel: Call Trace: > May 19 10:41:22 maverick kernel: > May 19 10:41:22 maverick kernel: ? __die+0x23/0x80 > May 19 10:41:22 maverick kernel: ? page_fault_oops+0x171/0x4e0 > May 19 10:41:22 maverick kernel: ? nf_queue+0x18/0x50 > May 19 10:41:22 maverick kernel: ? exc_page_fault+0x42c/0x730 > May 19 10:41:22 maverick kernel: ? asm_exc_page_fault+0x26/0x30 > May 19 10:41:22 maverick kernel: ? tcp_schedule_loss_probe+0x123/0x200 > May 19 10:41:22 maverick kernel: ? tcp_write_xmit+0x1eb/0x1330 > May 19 10:41:22 maverick kernel: ? tcp_sendmsg+0x2b/0x50 > May 19 10:41:22 maverick kernel: ? sock_write_iter+0x15e/0x190 > May 19 10:41:22 maverick kernel: ? vfs_write+0x3ab/0x450 > May 19 10:41:22 maverick kernel: ? ksys_write+0xc3/0xf0 > May 19 10:41:22 maverick kernel: ? do_syscall_64+0x5a/0x90 > May 19 10:41:22 maverick kernel: ? exit_to_user_mode_prepare+0x1a/0x140 > May 19 10:41:22 maverick kernel: ? syscall_exit_to_user_mode+0x2e/0x50 > May 19 10:41:22 maverick kernel: ? do_syscall_64+0x66/0x90 > May 19 10:41:22 maverick kernel: ? exit_to_user_mode_prepare+0x1a/0x140 > May 19 10:41:22 maverick kernel: ? syscall_exit_to_user_mode+0x2e/0x50 > May 19 10:41:22 maverick kernel: ? do_syscall_64+0x66/0x90 > May 19 10:41:22 maverick kernel: ? vfs_write+0x3ab/0x450 > May 19 10:41:22 maverick kernel: ? exit_to_user_mode_prepare+0x1a/0x140 > May 19 10:41:22 maverick kernel: ? syscall_exit_to_user_mode+0x2e/0x50 > May 19 10:41:22 maverick kernel: ? do_syscall_64+0x66/0x90 > May 19 10:41:22 maverick kernel: ? exit_to_user_mode_prepare+0x1a/0x140 > May 19 10:41:22 maverick kernel: ? syscall_exit_to_user_mode+0x2e/0x50 > May 19 10:41:22 maverick kernel: ? do_syscall_64+0x66/0x90 > May 19 10:41:22 maverick kernel: ? __hrtimer_run_queues+0x141/0x2b0 > May 19 10:41:22 maverick kernel: ? __pfx_read_tsc+0x10/0x10 > May 19 10:41:22 maverick kernel: ? ktime_get+0x43/0xb0 > May 19 10:41:22 maverick kernel: ? lapic_next_deadline+0x2c/0x50 > May 19 10:41:22 maverick kernel: ? clockevents_program_event+0x8d/0x100 > May 19 10:41:22 maverick kernel: ? hrtimer_interrupt+0x12b/0x250 > May 19 10:41:22 maverick kernel: ? exit_to_user_mode_prepare+0x1a/0x140 > May 19 10:41:22 maverick kernel: ? entry_SYSCALL_64_after_hwframe+0x78/0xe2 > May 19 10:41:22 maverick kernel: > May 19 10:41:22 maverick kernel: Modules linked in: esp4 tun act_mirred act= _connmark em_ipt act_gact cls_basic ifb sch_ingress xt_layer7 cls_u32 sch_htb= xt_NFQUEUE nfnetlink_queue xt_MASQUERADE pppoe pppox ppp_generic slhc 8021q = garp xt_time xt_set ip_set_hash_net xt_REDIRECT xt_connlimit nf_conncount xt_= multiport ip_set xt_owner xt_hashlimit xt_mac xt_policy xt_TCPMSS xt_conntrac= k xt_comment ipt_REJECT nf_reject_ipv4 xt_LOG xt_limit xt_mark xt_connmark nf= _log_syslog iptable_raw iptable_mangle iptable_filter vfat fat snd_hda_codec_= hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio i915 ax88796b = intel_rapl_common drm_buddy intel_powerclamp ttm coretemp drm_display_helper = kvm_intel drm_kms_helper sch_cake i2c_algo_bit kvm snd_hda_intel snd_intel_ds= pcfg snd_hda_codec at24 iTCO_wdt regmap_i2c snd_hda_core iTCO_vendor_support = asix mcs7830 snd_hwdep snd_pcm phylink usbnet snd_timer snd irqbypass mii i2c= _i801 r8169 lpc_ich intel_xhci_usb_role_switch roles realtek i2c_smbus soundc= ore pcspkr mfd_core rfkill_gp > May 19 10:41:22 maverick kernel: o rfkill > May 19 10:41:22 maverick kernel: intel_int0002_vgpio lp parport_pc parport= efivarfs crct10dif_pclmul crc32_pclmul polyval_generic i2c_hid_acpi i2c_hid = ghash_clmulni_intel sha512_ssse3 drm sha256_ssse3 sha1_ssse3 i2c_core video w= mi dm_mirror dm_region_hash dm_log dm_mod btrfs blake2b_generic xor lzo_compr= ess zstd_compress raid6_pq > May 19 10:41:22 maverick kernel: CR2: 0000000000000000 > May 19 10:41:22 maverick kernel: ---[ end trace 0000000000000000 ]--- > May 19 10:41:22 maverick kernel: RIP: 0010:0x0 > May 19 10:41:22 maverick kernel: Code: Unable to access opcode bytes at 0xf= fffffffffffffd6. > May 19 10:41:22 maverick kernel: RSP: 0018:ffffc90000433900 EFLAGS: 00010246 > May 19 10:41:22 maverick kernel: RAX: 0000000000000000 RBX: ffff88814cd371c= 0 RCX: 0000000000000000 > May 19 10:41:22 maverick kernel: RDX: 0000000000000000 RSI: ffffc900004339d= 8 RDI: 0000000000000000 > May 19 10:41:22 maverick kernel: RBP: 0000000000000218 R08: 000000000000000= 0 R09: 0000000000000000 > May 19 10:41:22 maverick kernel: R10: 0000000000000000 R11: 000000000000000= 0 R12: 0000000000000218 > May 19 10:41:22 maverick kernel: R13: ffff888101cf2d00 R14: 000000000000004= 0 R15: ffffc900004339d8 > May 19 10:41:22 maverick kernel: FS: 0000722d08168740(0000) GS:ffff88817bc= 00000(0000) knlGS:0000000000000000 > May 19 10:41:22 maverick kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008= 0050033 > May 19 10:41:22 maverick kernel: CR2: ffffffffffffffd6 CR3: 000000010a9ee00= 0 CR4: 00000000001006f0 > May 19 10:41:22 maverick kernel: note: tor[1585] exited with irqs disabled >=20 > I'm not sure what to make out of this, but it suggests that Core Update 186= needs a closer > look before it is ready to be released. >=20 > Thanks, and best regards, > Peter M=C3=BCller >=20 >> Hello development folks, >>=20 >> Core Update 186 (testing; see: https://www.ipfire.org/blog/ipfire-2-29-cor= e-update-186-is-available-for-testing) >> is running here for a couple of days by now without any major issues known= so far. >>=20 >> During the update, I merely noticed dracut complaining: >>=20 >>> dracut: Skipping program /bin/loginctl using in udev rule 71-seat.rules a= s it cannot be found >>=20 >> However, this does not appear to have any noticeable impact whatsoever. >>=20 >> The updated Lynis version now outputs significantly fewer warnings about d= eprecated >> grep parameters, which previously made output hard to read sometimes. >>=20 >> Tested IPFire functionalities in detail: >> - PPPoE dial-up via a DSL connection >> - IPsec (N2N connections only) >> - Squid (authentication enabled, using an upstream proxy) >> - OpenVPN (RW connections only) >> - IPS/Suricata (with Emerging Threats community ruleset enabled) >> - Guardian >> - Quality of Service >> - DNS (using DNS over TLS and strict QNAME minimisation) >> - Dynamic DNS >> - Tor (relay mode) >>=20 >> I am looking forward to the release of Core Update 186. >>=20 >> Thanks, and best regards, >> Peter M=C3=BCller > --===============4592867150562769305==--