Hello Matthias,

> On 19 Nov 2022, at 15:56, Matthias Fischer <matthias.fischer(a)ipfire.org> wrote:
> 
> Hi,
> 
> ...I'd like to have a small problem... ;-)
> 
> A few days ago, 'clamav 0.105.1' was updated, again:
> 
> =>
> https://blog.clamav.net/2022/11/second-clamav-100-release-candidate-and.html
> 
> "...[it] was intended to also include bug fixes for the jpeg and tiff
> Rust-based libraries that are bundled with the source code tarball.
> Unfortunately, those fixes were not all release-ready in time for the
> 0.105.1-2 packages."
> 
> So far, so [oh, forget it!].

This is *really* bad that they bundle so many libraries and make it very difficult for us to keep track of what vulnerabilities might be in clamav although they are part of a third-party library.

We should try to remove all of them and always build against the system libraries.

> Unfortunately, building the third version of 'clamav 0.105.1' with
> current 'next' failed:
> 
> ***SNIP***
> ...
>    error: package `tiff v0.8.0` cannot be built because it requires
> rustc 1.61.0 or newer, while the currently active rustc version is
> 1.60.0-nightly.
> 
>    [193/379] Building C object
> libclamav/CMakeFiles/lzma_sdk.dir/7z/7zIn.c.o
>    [194/379] Building C object
> libclamav/CMakeFiles/bytecode_runtime.dir/bytecode_nojit.c.o
>    [195/379] Building C object
> libclamav/CMakeFiles/yara.dir/yara_grammar.c.o
>    [196/379] Building C object libclamav/CMakeFiles/yara.dir/yara_lexer.c.o
>    yara_lexer.c:2571:24: warning: 'yy_fatal_error' defined but not used
> [-Wunused-function]
>    yara_lexer.c: In function 'yara_yylex':
>    yara_lexer.l:263:16: warning: '%s' directive output may be truncated
> writing up to 1023 bytes into a region of size 999 [-Wformat-truncation=]
>    In file included from /usr/include/stdio.h:906,
>    from yara_lexer.c:32:
>    /usr/include/bits/stdio2.h:54:10: note: '__builtin___snprintf_chk'
> output between 26 and 1049 bytes into a destination of size 1024
>    54 |   return __builtin___snprintf_chk (__s, __n,
> __USE_FORTIFY_LEVEL - 1,
>    |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>    55 |                                    __glibc_objsize (__s), __fmt,
>    |                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>    56 |                                    __va_arg_pack ());
>    |                                    ~~~~~~~~~~~~~~~~~
>    ninja: build stopped: subcommand failed.
>    make: *** [clamav:89: /usr/src/log/clamav-0.105.1] Error 1
> ***SNAP***

Great code quality. This is however not the reason why the build stopped. This is only a warning.

> Hm. Great.
> 
> So I tried the current 'rust 1.65' version.
> 
> This time, the building failed because of a rust component:
> 
> ***SNIP***
> ...
> Finished release [optimized] target(s) in 1.92s
>    cd /usr/src/cipher-0.3.0 &&         mkdir -pv
> "/usr/share/cargo/registry/cipher-0.3.0" && if
> CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo --offline
> metadata --format-version 1 --no-deps | jq -e
> ".packages[].targets[].kind | any(. == \"lib\")" | grep -q "true" ||
> CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo --offline
> metadata --format-version 1 --no-deps | jq -e
> ".packages[].targets[].kind | any(. == \"rlib\")" | grep -q "true" ||
> CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo --offline
> metadata --format-version 1 --no-deps | jq -e
> ".packages[].targets[].kind | any(. == \"proc-macro\")" | grep -q
> "true"; then awk
> '/^\\\[((.+\\\.)?((dev|build)-)?dependencies|features)/{f=1;next}
> /^\\\[/{f=0}; !f' < Cargo.toml > Cargo.toml.deps &&
> CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo --offline
> package -l | grep -wEv "Cargo.(lock|toml.orig)" | xargs -d "\n" cp -v
> --parents -a -t /usr/share/cargo/registry/cipher-0.3.0 && install -v -m
> 644 Cargo.toml.deps /usr/share/cargo/registry/cipher-0.3.0/Cargo.toml &&
> echo "{\"files\":{},\"package\":\"\"}" >
> /usr/share/cargo/registry/cipher-0.3.0/.cargo-checksum.json; fi && if
> true && CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo
> --offline metadata --format-version 1 --no-deps | jq -e
> ".packages[].targets[].kind | any(. == \"bin\")" | grep -q "true"; then
> CARGOPATH=/usr/src/cipher-0.3.0/.cargo RUSTC_BOOTSTRAP=1 cargo --offline
> install -Z avoid-dev-deps -j8 --no-track --path .; fi
>    mkdir: created directory '/usr/share/cargo/registry/cipher-0.3.0'
>    warning: No (git) VCS found for `/usr/src/cipher-0.3.0`
>    error: invalid inclusion of reserved file name Cargo.toml.orig in
> package source
>    cp: missing file operand
>    Try 'cp --help' for more information.
>    make: *** [rust-cipher:78: /usr/src/log/cipher-0.3.0] Error 123
> ***SNAP***

Rust is an absolute dependency hell. Ask Adolf and look at his latest patchset :)

> Ok, even greater.
> 
> Does anyone have an idea to solve this? I can't even find an updated
> package for , e.g., 'cipher-0.3.0tar.gz', although apparently I found at
> least an updated version (0.4.3) here:
> 
> => https://docs.rs/cipher/latest/cipher/#
> 
> But no download links... Hm! Where on earth did 'cipher-0.3.0.tar.gz'
> came from?

There is a little helper script in tools/ which you can use to automatically download the source and even generate an LFS file, because they all look the same:

https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=tools/download-rust-crate;h=f6a0fe035d30fdbddaa843ccac45251b0049088a;hb=HEAD

You can just run this as “tools/download-rust-crate cipher” and it should create everything you need. Just add it to make.sh and it should build.

> What makes me a bit nervous though is the fact that if clamav really can
> only be made to work with a major rust update, the other rust components
> might have to be updated as well. And I found 103 rust*-lfs files...

Yes. And every time we change one of those packages, we will have to ship *everything* that is related to Rust.

Such a great language. Stop using Rust, people.

-Michael

> 
> Any thoughts and hints welcome!
> 
> Best,
> Matthias