Re: [PATCH 1/2] vectorscan: Install vectorscan to replace hypersan

[Adolf Belka <adolf.belka@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 9978 bytes --]



On 29/05/2024 11:03, Michael Tremer wrote:
> 
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:128: CMakeFiles/hs_compile_shared.dir/src/compiler/compiler.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:408: CMakeFiles/hs_compile_shared.dir/src/nfa/goughcompile_reg.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:240: CMakeFiles/hs_compile_shared.dir/src/fdr/teddy_compile.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:142: CMakeFiles/hs_compile_shared.dir/src/compiler/error.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:310: CMakeFiles/hs_compile_shared.dir/src/nfa/accel_dfa_build_strat.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:296: CMakeFiles/hs_compile_shared.dir/src/hwlm/noodle_build.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:394: CMakeFiles/hs_compile_shared.dir/src/nfa/goughcompile_accel.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:324: CMakeFiles/hs_compile_shared.dir/src/nfa/accelcompile.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:254: CMakeFiles/hs_compile_shared.dir/src/fdr/teddy_engine_description.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:352: CMakeFiles/hs_compile_shared.dir/src/nfa/dfa_build_strat.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:100: CMakeFiles/hs_compile_shared.dir/src/hs.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:170: CMakeFiles/hs_compile_shared.dir/src/fdr/fdr_compile.cpp.o] Error 1
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:282: CMakeFiles/hs_compile_shared.dir/src/hwlm/hwlm_literal.cpp.o] Error 1
>     c++: error: unrecognized command-line option '-=power9'
>     make[3]: *** [CMakeFiles/hs_compile_shared.dir/build.make:366: CMakeFiles/hs_compile_shared.dir/src/nfa/dfa_min.cpp.o] Error 1
>     make[2]: *** [CMakeFiles/Makefile2:335: CMakeFiles/hs_compile_shared.dir/all] Error 2
>     make[1]: *** [Makefile:136: all] Error 2
>     make[1]: Leaving directory '/usr/src/vectorscan-vectorscan-5.4.11'
>     make: *** [vectorscan:79: /usr/src/log/vectorscan-vectorscan-5.4.11] Error 2
> 
> It looks like this package won’t compile on riscv64. Since this is not really a priority for us right now, we should disable the package for this architecture (I am sure hyperscan wouldn’t have compiled either) and wait for upstream to eventually provide support for this.
> 
> Any objections?

No objections from me.

Regards,
Adolf.
> 
> -Michael
> 
>> On 14 May 2024, at 16:17, Michael Tremer <michael.tremer(a)ipfire.org> wrote:
>>
>> Thank you for looking at this so quickly.
>>
>> Reviewed-by: Michael Tremer <michael.tremer(a)ipfire.org>
>>
>>> On 14 May 2024, at 11:39, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> - It has been announced that from hyperscan-5.5 onwards the licence for this package
>>>   will change from BSD tp proprietarty paid for version
>>> - This patch submission installs vectorscan whihc was created as a fork from hyperscan
>>>   andf that is being maintained and has indicated it will suay Open Source
>>> - Created new lfs file
>>> - Created nbew rootfile. This looks to match the hyperscan rootfile closely
>>> - Added vector scan to the make.sh file and removed hyperscan from it.
>>>
>>> Signed-off-by: Adolf Belka <adolf.belka(a)ipfire.org>
>>> ---
>>> config/rootfiles/common/vectorscan | 13 +++++
>>> lfs/vectorscan                     | 86 ++++++++++++++++++++++++++++++
>>> make.sh                            |  2 +-
>>> 3 files changed, 100 insertions(+), 1 deletion(-)
>>> create mode 100644 config/rootfiles/common/vectorscan
>>> create mode 100644 lfs/vectorscan
>>>
>>> diff --git a/config/rootfiles/common/vectorscan b/config/rootfiles/common/vectorscan
>>> new file mode 100644
>>> index 000000000..160dc3ae7
>>> --- /dev/null
>>> +++ b/config/rootfiles/common/vectorscan
>>> @@ -0,0 +1,13 @@
>>> +#usr/include/hs
>>> +#usr/include/hs/hs.h
>>> +#usr/include/hs/hs_common.h
>>> +#usr/include/hs/hs_compile.h
>>> +#usr/include/hs/hs_runtime.h
>>> +#usr/include/hs/hs_version.h
>>> +#usr/lib/libhs.so
>>> +usr/lib/libhs.so.5
>>> +usr/lib/libhs.so.5.4.11
>>> +#usr/lib/libhs_runtime.so
>>> +usr/lib/libhs_runtime.so.5
>>> +usr/lib/libhs_runtime.so.5.4.11
>>> +#usr/lib/pkgconfig/libhs.pc
>>> diff --git a/lfs/vectorscan b/lfs/vectorscan
>>> new file mode 100644
>>> index 000000000..0b60e19c9
>>> --- /dev/null
>>> +++ b/lfs/vectorscan
>>> @@ -0,0 +1,86 @@
>>> +###############################################################################
>>> +#                                                                             #
>>> +# IPFire.org - A linux based firewall                                         #
>>> +# Copyright (C) 2007-2024  IPFire Team  <info(a)ipfire.org>                     #
>>> +#                                                                             #
>>> +# This program is free software: you can redistribute it and/or modify        #
>>> +# it under the terms of the GNU General Public License as published by        #
>>> +# the Free Software Foundation, either version 3 of the License, or           #
>>> +# (at your option) any later version.                                         #
>>> +#                                                                             #
>>> +# This program is distributed in the hope that it will be useful,             #
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of              #
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the               #
>>> +# GNU General Public License for more details.                                #
>>> +#                                                                             #
>>> +# You should have received a copy of the GNU General Public License           #
>>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.       #
>>> +#                                                                             #
>>> +###############################################################################
>>> +
>>> +###############################################################################
>>> +# Definitions
>>> +###############################################################################
>>> +
>>> +include Config
>>> +
>>> +VER        = 5.4.11
>>> +
>>> +THISAPP    = vectorscan-vectorscan-$(VER)
>>> +DL_FILE    = $(THISAPP).tar.gz
>>> +DL_FROM    = $(URL_IPFIRE)
>>> +DIR_APP    = $(DIR_SRC)/$(THISAPP)
>>> +TARGET     = $(DIR_INFO)/$(THISAPP)
>>> +
>>> +# The compiler uses a lot of memory to compile hyperscan, hence we reduce
>>> +# the total number of processes a little bit to be able to build on
>>> +# smaller machines
>>> +MAX_PARALLELISM = $(shell echo $$(( $(SYSTEM_MEMORY) / 1024)))
>>> +
>>> +###############################################################################
>>> +# Top-level Rules
>>> +###############################################################################
>>> +
>>> +objects = $(DL_FILE)
>>> +
>>> +$(DL_FILE) = $(DL_FROM)/$(DL_FILE)
>>> +
>>> +$(DL_FILE)_BLAKE2 = a8f5a1230af0ddf7d9fb9299769ec1736d37ac3284f6a98b1e650af461206cf459eac35d13a47beb6683786c6529539b2d082edf426e7d4890ed11804c76268b
>>> +
>>> +install : $(TARGET)
>>> +
>>> +check : $(patsubst %,$(DIR_CHK)/%,$(objects))
>>> +
>>> +download :$(patsubst %,$(DIR_DL)/%,$(objects))
>>> +
>>> +b2 : $(subst %,%_BLAKE2,$(objects))
>>> +
>>> +###############################################################################
>>> +# Downloading, checking, b2sum
>>> +###############################################################################
>>> +
>>> +$(patsubst %,$(DIR_CHK)/%,$(objects)) :
>>> + @$(CHECK)
>>> +
>>> +$(patsubst %,$(DIR_DL)/%,$(objects)) :
>>> + @$(LOAD)
>>> +
>>> +$(subst %,%_BLAKE2,$(objects)) :
>>> + @$(B2SUM)
>>> +
>>> +###############################################################################
>>> +# Installation Details
>>> +###############################################################################
>>> +
>>> +$(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects))
>>> + @$(PREBUILD)
>>> + @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar axf $(DIR_DL)/$(DL_FILE)
>>> + cd $(DIR_APP) && cmake . \
>>> + -DCMAKE_INSTALL_PREFIX:PATH=/usr \
>>> + -DBUILD_SHARED_LIBS=ON \
>>> + -DCMAKE_BUILD_TYPE=RELEASE \
>>> + -DBUILD_EXAMPLES=OFF
>>> + cd $(DIR_APP) && make $(MAKETUNING)
>>> + cd $(DIR_APP) && make install
>>> + @rm -rf $(DIR_APP)
>>> + @$(POSTBUILD)
>>> diff --git a/make.sh b/make.sh
>>> index a4ba09326..9977e762b 100755
>>> --- a/make.sh
>>> +++ b/make.sh
>>> @@ -1402,7 +1402,7 @@ buildipfire() {
>>>   lfsmake2 libhtp
>>>   lfsmake2 colm
>>>   lfsmake2 ragel
>>> -  lfsmake2 hyperscan
>>> +  lfsmake2 vectorscan
>>>   lfsmake2 suricata
>>>   lfsmake2 ids-ruleset-sources
>>>   lfsmake2 ipblocklist-sources
>>> -- 
>>> 2.44.0
>>>
>>
> 

-- 
Sent from my laptop