Re: Updating rust and eco system

[Michael Tremer <michael.tremer@ipfire.org>]

Hello Adolf,

> On 23 Jan 2026, at 11:06, Adolf Belka <adolf.belka@ipfire.org> wrote:
> 
> Hi Michael,
> 
> On 23/01/2026 11:31, Michael Tremer wrote:
>> Hello Stefan,
>> Hello list,
>> Thank you for looking at this. Of course it is very important that we are able to stay on the latest version of Suricata.
>> I have merged your monster of a patch so that we can move on for now, but I have a couple of bigger questions that we all should have a look at:
>> Adolf has in the past spent a lot of time on updating Rust. This is all tapping into Python - or rather python-cryptography - having some Rust code that has further dependencies. In essence, it has been a huge headache to update this. Maybe Adolf even has some other words for this all.
> 
> My words on this are that I have now tried multiple times to get a new python update built. Each time I have done it a bit different but the end result has been the same and that is that python-cryptography (which requires rust modules to be built) ends up requiring python-maturin that requires more rust modules but at the end of this the python-cryptography fails to find the built rust modules.
> 
> I have been stuck at this last point so many times that I have realised that I am finding lots of reasons not to go and work on the python update.
> That is not a good position and also python has now moved from 3.13 to 3.14 so things are moving away from me.
> 
> I have come to the conclusion that someone else, more capable than me needs to have a go at the python update, so I am giving up on it but will continue working on other things.

Hmm okay, you sound like you are giving up on this :) I know how many hours (we probably need to measure those in days or even weeks) you have spent on this though.

Let’s pool resources together and finally get this done. Hopefully this will be a smoother ride as a combined effort.

>> Just building cbindgen has required a further ~98 Rust crates to be packaged. Often we have the same crate in different versions because other crates have pinned a specific version. In total, we currently have ~790 packages in IPFire. Out of those, there are 202 packages in the rust-* namespace. That is pretty much a quarter of the distribution. Although not a lot in size, this is a considerable maintenance burden.
>> ClamAV and Suricata have (recently?) started to bundle all their Rust dependencies with their release tarballs. Although this is not a good thing for many other reasons, it will move the onus onto the upstream projects to provide whatever they need. If their dependencies (and the dependencies of their dependencies) explode, this is not really our problem any more as well as any supply chain problems. Great - within reason.
>> That leaves us with only very few packages that would actually require any external Rust crates (Suricata is even configured to *exclusively* use their bundled crates): cbindgen as a new thing, python-cryptography, anything else? We might actually only need a fraction of the Rust crates that we currently have as the only packages that may actually tap into our locally built repository are only those two.
> 
> Unfortunately there is the addon oci-python-sdk that uses python-cryptography.

python-cryptography was on my list. oci-python-sdk only uses Rust indirectly through python-cryptography, right?

>> Is anyone happy to give this all a try and cleanup any old Rust deps? That way, I hope we will have a much smoother ride moving forward with a Python update.
> 
> I can take the current status, before Stefan's patches, and see how many existing rust modules can be removed. Anything that can be removed is a step forward.

Yes, I think we should try to shrink what we have now if that is possible at all. As most packages are bundling all Rust deps, there should be some we won’t need any more in the system.

Then, we hopefully have much less to update/worry about in any other way when we start touching python-cryptography.

So who is volunteering to do this? Commenting out all Rust packages, then build python-cryptography which will fail as it requires some Rust crates. Those will be there so they will only have to be commented in again. Once the package builds, we should then have a couple of packages still commented that we can drop.

> I think a problem moving forward is that more python modules are ending up being a combination of python and rust as the cryptography and maturin modules have already done. I have also seen a lot of rust modules covering the same stuff as covered by python modules. So the future I think looks like it will continue to be very frustrating.

Yes it does, but we will have to find a way whether we want it or not.

-Michael

> Regards,
> 
> Adolf.
> 
> 
>> All the best,
>> -Michael
>>> On 22 Jan 2026, at 17:38, Stefan Schantl <stefan.schantl@ipfire.org> wrote:
>>> 
>>> Hello list followers,
>>> 
>>> I'm currently updating rust and affected modules.
>>> 
>>> This happends mainly because I'm trying to fix the "suricata cache
>>> grows infinite" problem, which a lot of people are affected.
>>> 
>>> To archive this, I ported the patches from suricata main development
>>> branch to our used suricata version (8.0.3).
>>> 
>>> To perform a full build, a new tool called cbindgen - which is a rust
>>> to c bindings generator, is required.
>>> 
>>> Sadly this tool is also written in rust and requires some new
>>> dependencies and a more up to date rust compiler.
>>> 
>>> I hope to send a patchset for all this very soon to the mailing list.
>>> 
>>> Best regards,
>>> 
>>> -Stefan