From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.haj.ipfire.org (localhost [IPv6:::1]) by mail02.haj.ipfire.org (Postfix) with ESMTP id 4dzfKl2fk2z2yBj for ; Sun, 25 Jan 2026 17:48:43 +0000 (UTC) Received: from mail01.ipfire.org (mail01.haj.ipfire.org [172.28.1.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature ECDSA (secp384r1 raw public key) server-digest SHA384 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail01.haj.ipfire.org", Issuer "R12" (verified OK)) by mail02.haj.ipfire.org (Postfix) with ESMTPS id 4dzfKg6sWmz2xHh for ; Sun, 25 Jan 2026 17:48:39 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519) (No client certificate requested) by mail01.ipfire.org (Postfix) with ESMTPSA id 4dzfKg0nJxz3s4 for ; Sun, 25 Jan 2026 17:48:39 +0000 (UTC) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003ed25519; t=1769363319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yaUbdoc3PcE9WEV/wYfmb3kcoy1Cmcm71dTp2YVHW8U=; b=amBxJHSloAKN9AJ4oWVYY/ZfcPmVSJyiU7xbEXzcfcQxxv67TOlq21SZyW9Xp9/Op2wRr+ 94m/PstZTy29w/Bw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfire.org; s=202003rsa; t=1769363319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yaUbdoc3PcE9WEV/wYfmb3kcoy1Cmcm71dTp2YVHW8U=; b=vCEadUzdRGbTwRc5qi3bgwzArktcOFn2RC0b2jMFHvoR8MfUWSrU5m3ztfsCN+Jgzvi97Z NUqf3qc+YunLDdsDkEBw4Dbum38mq4qyF2VeTZl5N2T3yPuf7o0nh6R6rcBIQzT7PGymlU sfBqgJh9Z0rqBBcQcywxhhWg0rvwra382TySuDxGZ4k0UrIhsvIcQsbYq2Gy1fFEptZKhM UTiNQzP3cSnIIhyrziehAa2b5inyv/4hRTBnqNq3+xpZ7GqSQn7w6x78aVmUFXr9t7HxZc dLY+f+4tcSyU5Y2ehoEzodOJ997nRrBjeZAln8MyRP/cgLPBACwAray02S2Ytw== Message-ID: Subject: Re: Updating rust and eco system From: Stefan Schantl To: development@lists.ipfire.org Date: Sun, 25 Jan 2026 18:46:21 +0100 In-Reply-To: <2F324FA8-89B4-4EDC-A9F4-95DBB0E11CF6@ipfire.org> References: <02AF1D50-1E51-48DE-A5EE-D89C89B3B34E@ipfire.org> <0772cd37-21e8-45c0-9543-957c4688b56d@ipfire.org> <2F324FA8-89B4-4EDC-A9F4-95DBB0E11CF6@ipfire.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: list List-Id: List-Subscribe: , List-Unsubscribe: , List-Post: List-Help: Sender: Mail-Followup-To: MIME-Version: 1.0 Hello Adolf, Hello Michael, I would give the rust cleanup a try in the next few days. Adolf may I can ask you to put your current state of the python update into a git repositry? Thanks in advance, -Stefan > Hello Adolf, >=20 > > On 23 Jan 2026, at 11:06, Adolf Belka > > wrote: > >=20 > > Hi Michael, > >=20 > > On 23/01/2026 11:31, Michael Tremer wrote: > > > Hello Stefan, > > > Hello list, > > > Thank you for looking at this. Of course it is very important > > > that we are able to stay on the latest version of Suricata. > > > I have merged your monster of a patch so that we can move on for > > > now, but I have a couple of bigger questions that we all should > > > have a look at: > > > Adolf has in the past spent a lot of time on updating Rust. This > > > is all tapping into Python - or rather python-cryptography - > > > having some Rust code that has further dependencies. In essence, > > > it has been a huge headache to update this. Maybe Adolf even has > > > some other words for this all. > >=20 > > My words on this are that I have now tried multiple times to get a > > new python update built. Each time I have done it a bit different > > but the end result has been the same and that is that python- > > cryptography (which requires rust modules to be built) ends up > > requiring python-maturin that requires more rust modules but at the > > end of this the python-cryptography fails to find the built rust > > modules. > >=20 > > I have been stuck at this last point so many times that I have > > realised that I am finding lots of reasons not to go and work on > > the python update. > > That is not a good position and also python has now moved from 3.13 > > to 3.14 so things are moving away from me. > >=20 > > I have come to the conclusion that someone else, more capable than > > me needs to have a go at the python update, so I am giving up on it > > but will continue working on other things. >=20 > Hmm okay, you sound like you are giving up on this :) I know how many > hours (we probably need to measure those in days or even weeks) you > have spent on this though. >=20 > Let=E2=80=99s pool resources together and finally get this done. Hopefull= y > this will be a smoother ride as a combined effort. >=20 > > > Just building cbindgen has required a further ~98 Rust crates to > > > be packaged. Often we have the same crate in different versions > > > because other crates have pinned a specific version. In total, we > > > currently have ~790 packages in IPFire. Out of those, there are > > > 202 packages in the rust-* namespace. That is pretty much a > > > quarter of the distribution. Although not a lot in size, this is > > > a considerable maintenance burden. > > > ClamAV and Suricata have (recently?) started to bundle all their > > > Rust dependencies with their release tarballs. Although this is > > > not a good thing for many other reasons, it will move the onus > > > onto the upstream projects to provide whatever they need. If > > > their dependencies (and the dependencies of their dependencies) > > > explode, this is not really our problem any more as well as any > > > supply chain problems. Great - within reason. > > > That leaves us with only very few packages that would actually > > > require any external Rust crates (Suricata is even configured to > > > *exclusively* use their bundled crates): cbindgen as a new thing, > > > python-cryptography, anything else? We might actually only need a > > > fraction of the Rust crates that we currently have as the only > > > packages that may actually tap into our locally built repository > > > are only those two. > >=20 > > Unfortunately there is the addon oci-python-sdk that uses python- > > cryptography. >=20 > python-cryptography was on my list. oci-python-sdk only uses Rust > indirectly through python-cryptography, right? >=20 > > > Is anyone happy to give this all a try and cleanup any old Rust > > > deps? That way, I hope we will have a much smoother ride moving > > > forward with a Python update. > >=20 > > I can take the current status, before Stefan's patches, and see how > > many existing rust modules can be removed. Anything that can be > > removed is a step forward. >=20 > Yes, I think we should try to shrink what we have now if that is > possible at all. As most packages are bundling all Rust deps, there > should be some we won=E2=80=99t need any more in the system. >=20 > Then, we hopefully have much less to update/worry about in any other > way when we start touching python-cryptography. >=20 > So who is volunteering to do this? Commenting out all Rust packages, > then build python-cryptography which will fail as it requires some > Rust crates. Those will be there so they will only have to be > commented in again. Once the package builds, we should then have a > couple of packages still commented that we can drop. >=20 > > I think a problem moving forward is that more python modules are > > ending up being a combination of python and rust as the > > cryptography and maturin modules have already done. I have also > > seen a lot of rust modules covering the same stuff as covered by > > python modules. So the future I think looks like it will continue > > to be very frustrating. >=20 > Yes it does, but we will have to find a way whether we want it or > not. >=20 > -Michael >=20 > > Regards, > >=20 > > Adolf. > >=20 > >=20 > > > All the best, > > > -Michael > > > > On 22 Jan 2026, at 17:38, Stefan Schantl > > > > wrote: > > > >=20 > > > > Hello list followers, > > > >=20 > > > > I'm currently updating rust and affected modules. > > > >=20 > > > > This happends mainly because I'm trying to fix the "suricata > > > > cache > > > > grows infinite" problem, which a lot of people are affected. > > > >=20 > > > > To archive this, I ported the patches from suricata main > > > > development > > > > branch to our used suricata version (8.0.3). > > > >=20 > > > > To perform a full build, a new tool called cbindgen - which is > > > > a rust > > > > to c bindings generator, is required. > > > >=20 > > > > Sadly this tool is also written in rust and requires some new > > > > dependencies and a more up to date rust compiler. > > > >=20 > > > > I hope to send a patchset for all this very soon to the mailing > > > > list. > > > >=20 > > > > Best regards, > > > >=20 > > > > -Stefan >=20 >=20