From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4f097q2Jrhz332Z for ; Mon, 26 Jan 2026 13:56:55 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [IPv6:2001:678:b28::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1 raw public key) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R12" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4f097l6C1dz2xLw for ; Mon, 26 Jan 2026 13:56:51 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4f097j07tRz5fY for ; Mon, 26 Jan 2026 13:56:48 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1769435809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=78WWxKncwCRnqdVMCEtE/kgxmjW1eWQMLt0Y20cWFXw=; b=/YF2wfLgTJ4LTMbtpvnbOuqiH81d9u/jNg0Z0WHL0Op9yGNua6c/8M448AB/zPhi+UHcW3 3EO9P498pyBC15BA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1769435809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=78WWxKncwCRnqdVMCEtE/kgxmjW1eWQMLt0Y20cWFXw=; b=UTTyCNS1BPXwjeZmV32u7oepcfixe9DPw0hKo5Qz4u7X1yH5Zc2qPKaNVnVv2yGn8fdUJy oyI1wVL2QEtTuw0pkvNyj/Arpw5yguiL/IBVsEJgkDCp5VHoEp0DWgwJIMm5kP8k79n5hF 99ATspRrWdD9Dom4gv7hAWrVjOHMhHnhzvGA+8+3bq4rPVazcj33tjz9oHY01QEqa46kwr f6luLQ2I9UG3XW9vLQlCTYg24SKxJrd+pjYv9LgYXbB2M6oE+FH5m2in9ntSkpomC0mBsd d328/3x5uUSf3GdKzIyiTjBAX/UVehLpC28PGIxarORhtsOEAs0/ZxWCimcL4g== Message-ID: <49cc50bd757ca86f2f744086705935ffd04a6604.camel@ipfire.org> Subject: Re: Updating rust and eco system From: Stefan Schantl To: development@lists.ipfire.org Date: Mon, 26 Jan 2026 14:54:29 +0100 In-Reply-To: References: <02AF1D50-1E51-48DE-A5EE-D89C89B3B34E@ipfire.org> <0772cd37-21e8-45c0-9543-957c4688b56d@ipfire.org> <2F324FA8-89B4-4EDC-A9F4-95DBB0E11CF6@ipfire.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: MIME-Version: 1.0 Hello list, currently I'm working on cleaning up the rust packages. For these I disabled all rust modules in the make.sh file and perform a clean build as Michael suggested. At the moment I'm past the stage where "cbindgen" successfully has been build and have 103 rust modules (inlcluding there sub-dependencies) only for this one tool. An additional rust module is required to build suricata. This is because of patching the source code the required rust module is not part of their source tarball. This currently summs to 104 rust modules for the moment. I'm looking forward when python-cryptography kicks in its module whishes.... Best regards, -Stefan > Hello Adolf, > Hello Michael, >=20 > I would give the rust cleanup a try in the next few days. >=20 > Adolf may I can ask you to put your current state of the python > update > into a git repositry? >=20 > Thanks in advance, >=20 > -Stefan >=20 > > Hello Adolf, > >=20 > > > On 23 Jan 2026, at 11:06, Adolf Belka > > > wrote: > > >=20 > > > Hi Michael, > > >=20 > > > On 23/01/2026 11:31, Michael Tremer wrote: > > > > Hello Stefan, > > > > Hello list, > > > > Thank you for looking at this. Of course it is very important > > > > that we are able to stay on the latest version of Suricata. > > > > I have merged your monster of a patch so that we can move on > > > > for > > > > now, but I have a couple of bigger questions that we all should > > > > have a look at: > > > > Adolf has in the past spent a lot of time on updating Rust. > > > > This > > > > is all tapping into Python - or rather python-cryptography - > > > > having some Rust code that has further dependencies. In > > > > essence, > > > > it has been a huge headache to update this. Maybe Adolf even > > > > has > > > > some other words for this all. > > >=20 > > > My words on this are that I have now tried multiple times to get > > > a > > > new python update built. Each time I have done it a bit different > > > but the end result has been the same and that is that python- > > > cryptography (which requires rust modules to be built) ends up > > > requiring python-maturin that requires more rust modules but at > > > the > > > end of this the python-cryptography fails to find the built rust > > > modules. > > >=20 > > > I have been stuck at this last point so many times that I have > > > realised that I am finding lots of reasons not to go and work on > > > the python update. > > > That is not a good position and also python has now moved from > > > 3.13 > > > to 3.14 so things are moving away from me. > > >=20 > > > I have come to the conclusion that someone else, more capable > > > than > > > me needs to have a go at the python update, so I am giving up on > > > it > > > but will continue working on other things. > >=20 > > Hmm okay, you sound like you are giving up on this :) I know how > > many > > hours (we probably need to measure those in days or even weeks) you > > have spent on this though. > >=20 > > Let=E2=80=99s pool resources together and finally get this done. Hopefu= lly > > this will be a smoother ride as a combined effort. > >=20 > > > > Just building cbindgen has required a further ~98 Rust crates > > > > to > > > > be packaged. Often we have the same crate in different versions > > > > because other crates have pinned a specific version. In total, > > > > we > > > > currently have ~790 packages in IPFire. Out of those, there are > > > > 202 packages in the rust-* namespace. That is pretty much a > > > > quarter of the distribution. Although not a lot in size, this > > > > is > > > > a considerable maintenance burden. > > > > ClamAV and Suricata have (recently?) started to bundle all > > > > their > > > > Rust dependencies with their release tarballs. Although this is > > > > not a good thing for many other reasons, it will move the onus > > > > onto the upstream projects to provide whatever they need. If > > > > their dependencies (and the dependencies of their dependencies) > > > > explode, this is not really our problem any more as well as any > > > > supply chain problems. Great - within reason. > > > > That leaves us with only very few packages that would actually > > > > require any external Rust crates (Suricata is even configured > > > > to > > > > *exclusively* use their bundled crates): cbindgen as a new > > > > thing, > > > > python-cryptography, anything else? We might actually only need > > > > a > > > > fraction of the Rust crates that we currently have as the only > > > > packages that may actually tap into our locally built > > > > repository > > > > are only those two. > > >=20 > > > Unfortunately there is the addon oci-python-sdk that uses python- > > > cryptography. > >=20 > > python-cryptography was on my list. oci-python-sdk only uses Rust > > indirectly through python-cryptography, right? > >=20 > > > > Is anyone happy to give this all a try and cleanup any old Rust > > > > deps? That way, I hope we will have a much smoother ride moving > > > > forward with a Python update. > > >=20 > > > I can take the current status, before Stefan's patches, and see > > > how > > > many existing rust modules can be removed. Anything that can be > > > removed is a step forward. > >=20 > > Yes, I think we should try to shrink what we have now if that is > > possible at all. As most packages are bundling all Rust deps, there > > should be some we won=E2=80=99t need any more in the system. > >=20 > > Then, we hopefully have much less to update/worry about in any > > other > > way when we start touching python-cryptography. > >=20 > > So who is volunteering to do this? Commenting out all Rust > > packages, > > then build python-cryptography which will fail as it requires some > > Rust crates. Those will be there so they will only have to be > > commented in again. Once the package builds, we should then have a > > couple of packages still commented that we can drop. > >=20 > > > I think a problem moving forward is that more python modules are > > > ending up being a combination of python and rust as the > > > cryptography and maturin modules have already done. I have also > > > seen a lot of rust modules covering the same stuff as covered by > > > python modules. So the future I think looks like it will continue > > > to be very frustrating. > >=20 > > Yes it does, but we will have to find a way whether we want it or > > not. > >=20 > > -Michael > >=20 > > > Regards, > > >=20 > > > Adolf. > > >=20 > > >=20 > > > > All the best, > > > > -Michael > > > > > On 22 Jan 2026, at 17:38, Stefan Schantl > > > > > wrote: > > > > >=20 > > > > > Hello list followers, > > > > >=20 > > > > > I'm currently updating rust and affected modules. > > > > >=20 > > > > > This happends mainly because I'm trying to fix the "suricata > > > > > cache > > > > > grows infinite" problem, which a lot of people are affected. > > > > >=20 > > > > > To archive this, I ported the patches from suricata main > > > > > development > > > > > branch to our used suricata version (8.0.3). > > > > >=20 > > > > > To perform a full build, a new tool called cbindgen - which > > > > > is > > > > > a rust > > > > > to c bindings generator, is required. > > > > >=20 > > > > > Sadly this tool is also written in rust and requires some new > > > > > dependencies and a more up to date rust compiler. > > > > >=20 > > > > > I hope to send a patchset for all this very soon to the > > > > > mailing > > > > > list. > > > > >=20 > > > > > Best regards, > > > > >=20 > > > > > -Stefan > >=20 > >=20