Re: Failure building collectd when autoconf has been updated to 2.72

[Adolf Belka <adolf.belka@ipfire.org>]

[-- Attachment #1: Type: text/plain, Size: 13602 bytes --]

Hi Michael,


On 25/11/2024 19:25, Adolf Belka wrote:
> Hi Michael,
>
> On 23/11/2024 13:09, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 23/11/2024 10:12, Michael Tremer wrote:
>>> Hello Adolf,
>>>
>>> Hmm, then let us maybe look at migrating collectd. It will be a 
>>> pain, but it seems that fixing the autoconf problem won’t be much 
>>> easier.
>>>
>>> There is a migration script here:
>>>
>>> https://github.com/collectd/collectd/blob/db4a13bdf37b1103c2014261458323e26b034eec/contrib/migrate-4-5.px#L107-L108 
>>>
>>>
>>> Should we maybe start with building a new branch with the latest 
>>> version of collectd 5 and simply check if a newly installed system 
>>> is still working? I don’t think there will be that many changes 
>>> required. Would you be up for giving that a go?
>>
>> I will give that a go and see what happens and we can work from there.
>>
> I gave it a go and have been very successful.
>
> Building the collectd-5.12.0 did result in various build failures. I 
> think I had to do around 7 build attempts but I progressively moved 
> forward and eventually succeeded in a successful build of the new 
> collectd and then the rest of the IPFire build.
>
> I then installed the CU190 iso with collectd-5.12.0 into a vm system. 
> The fresh install already updates all the rrd files with the updated 
> changes from 4.x to 5.x
>
> There were a range of graphs that failed due to, for example, a DS 
> named ping not being found in the ping rrd file.
>
> Checking the ping rrd file with rrdtool info I found that in 5.x the 
> ping rrd file DS is now "value" in place of "ping".
>
> I then found the DEF statement in graphs.pl for the ping graph and 
> changed the rrd:ping section to rrd:value and the graph started 
> operating and collecting data. I also did it with the conntrack entry 
> which changed from "entropy" to "value" and again the graph started 
> working.
>
> So I know now what I need to do - find all the places where the DS has 
> changed in an rrd file and correct those in the IPFire code.
>
> I tested out the migrate-4-5.px file and in its first phase it creates 
> a script which will update the DS values to their new versions. I can 
> use the output of that script to identify which files have got a 
> change in them, find where in the code that file is accessed and 
> modify it.
>
> We will also need to use the migrate script in the restore portion of 
> the backup process as the rrd files are in the backup and restoring 
> them will give files with a DS of "ping" for the ping rrd one, which 
> will not be recognised with the modified DS values in graphs.pl etc.
>
> Running the migrate script after doing a restore will convert all the 
> DS values to their 5.x values. If the entries in the backup are 
> already updated then the rrdtool tune command run by the migrate 
> script will give the error message
>
> ERROR: unknown data source name 'ping'
>
> in the example of the ping rrd file but in that case the entry is left 
> as it is so the migrate script should be able to work on restores with 
> the 4.x and 5.x rrd values.
>
> It looks like both DS names and DS types have been changed for some of 
> the plugins.
>
>
> Anyway, things are looking very promising. Should be able to look at 
> testing this out in unstable early in the new year.
>
So it turned out to be much easier than any of us thought. The changes 
needed in graphs.pl are only 4 lines to be changed.
The interface directory becomes the interface-$interface directory and 
there are two lines for that and then the rrd:ping (gateway graph) and 
rrd:entropy (conntrack graph) entries need to both be changed to rrd:value.

I also added the migrate-4-5 script into backup.pl

Tested all above out on my vm testbed. Fresh install of IPFire gave the 
new names and directories as required and everything worked. I also then 
restored an older backup that had the old names etc and the code in 
backup.pl also worked without any hitch. The graphs all worked.

The only thing I haven't done yet is adding the migrate code I put into 
backup.pl into the update.sh script. I will do that when next is updated 
to become CU191.

I will also just confirm that with the new collectd is happy to build 
with autoconf-2.72. Everything I have done so far is with the existing 
autoconf-2.71.

I think that the collectd update should be good to go into Core Update 
192 early in the new year.

Regards,

Adolf.

>
> Regards,
>
> Adolf.
>
>> Regards,
>>
>> Adolf
>>
>>>
>>> If we can then run the migration script on existing systems we might 
>>> almost be there…
>>>
>>> -Michael
>>>
>>>> On 22 Nov 2024, at 18:01, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> Hi Michael,
>>>>
>>>> On 20/11/2024 19:25, Michael Tremer wrote:
>>>>> Let’s maybe try to avoid running autoconf altogether.
>>>>> There might be a small chance because only two patches change 
>>>>> configure.ac. There are others that change Makefile.in, so this 
>>>>> might not work at all. Let’s see…
>>>>> Could you please remove the patches in src/patches/collectd with 
>>>>> number 12 and 13? Then remove the autoreconf commands from 
>>>>> lfs/collectd.
>>>>> Things should hopefully still apply and after the first run of 
>>>>> configure, make should not try to run autoconf again. Not sure but 
>>>>> worth a try.
>>>>
>>>> I removed the two patches and also removed the two autoreconf 
>>>> commands, leaving the autoupdate command in place.
>>>>
>>>> Unfortunately it failed to build and had the following error message
>>>>
>>>> aclocal.m4:16: warning: this file was generated for autoconf 2.67.
>>>> You have another version of autoconf.  It may work, but is not 
>>>> guaranteed to.
>>>> If you have problems, you may need to regenerate the build system 
>>>> entirely.
>>>> To do so, use the procedure documented by the package, typically 
>>>> `autoreconf'.
>>>> configure.in:533: error: AC_PROG_CC cannot be called after 
>>>> AM_PROG_CC_C_O
>>>> configure.in:533: the top level
>>>> autom4te: error: /tools_x86_64/bin/m4 failed with exit status: 1
>>>> make[1]: *** [Makefile:375: configure] Error 1
>>>> make[1]: Leaving directory '/usr/src/collectd-4.10.9'
>>>> make: *** [collectd:117: /usr/src/log/collectd-4.10.9] Error 2
>>>>
>>>>> I can’t remember what has stopped us from moving with upstream 
>>>>> apart from the RRD file format. If this all doesn’t work, maybe we 
>>>>> should look at this all again and see if we can’t migrate to the 
>>>>> latest upstream version again.
>>>>
>>>> As far as I can remember it was mentioned that the issue was the 
>>>> changed format and RRD being used in so many places making the 
>>>> update require changes in so many places in IPFire-2.x
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>>
>>>>> Best,
>>>>> -Michael
>>>>>> On 19 Nov 2024, at 13:55, Adolf Belka <adolf.belka(a)ipfire.org> 
>>>>>> wrote:
>>>>>> Hi Michael,
>>>>>>
>>>>>> Also reran the collectd autoreconf with the autoconf-2.71 and in 
>>>>>> that case the line has yes:) so something about the autoconf-2.72 
>>>>>> is causing that one yes: in the configure file to be missing the 
>>>>>> ) . There are something like 20 of those yes: entries and with 
>>>>>> the autoconf-2.72 only the one I show below is the one with a 
>>>>>> missing ), all the rest have a yes:) entry.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Adolf.
>>>>>>
>>>>>>> On 19/11/2024 14:43, Adolf Belka wrote:
>>>>>>> Hi Michael,
>>>>>>>
>>>>>>> I obtained the configure created by the autoreconf etc and I 
>>>>>>> have found the error. Here is the affected line 18710
>>>>>>>
>>>>>>>
>>>>>>> 18706   if test "x$ac_ct_CC" = x; then
>>>>>>> 18707     CC=""
>>>>>>> 18708   else
>>>>>>> 18709     case $cross_compiling:$ac_tool_warned in
>>>>>>> 18710 yes:
>>>>>>> 18711
>>>>>>> 18712 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: WARNING: 
>>>>>>> using cross tools not prefixed with host triplet" >&5
>>>>>>> 18713 printf "%s\n" "$as_me: WARNING: using cross tools not 
>>>>>>> prefixed with host triplet" >&2;}
>>>>>>> 18714 ac_tool_warned=yes ;;
>>>>>>> 18715 esac
>>>>>>> 18716     CC=$ac_ct_CC
>>>>>>> 18717   fi
>>>>>>> 18718 else
>>>>>>> 18719   CC="$ac_cv_prog_CC"
>>>>>>> 18720 fi
>>>>>>>
>>>>>>> That line should be yes:) so it is missing a right bracket. Not 
>>>>>>> sure if you can figure where and why that is occurring but 
>>>>>>> presumably a patch can be written to be applied after the 
>>>>>>> autoreconf etc and before the ./configure command
>>>>>>>
>>>>>>> Regards,
>>>>>>> Adolf.
>>>>>>>
>>>>>>>
>>>>>>>> On 19/11/2024 12:41, Michael Tremer wrote:
>>>>>>>> Hello Adolf,
>>>>>>>> In the build script we are running autoreconf which will 
>>>>>>>> regenerate the configure script:
>>>>>>>> https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=lfs/collectd;h=d1d4ea721386803c31599315de956373417c2dcf;hb=HEAD#l119 
>>>>>>>>
>>>>>>>> Is there any output of that command? There should be some 
>>>>>>>> warnings which might help us to find out what we need to change.
>>>>>>>>> On 13 Nov 2024, at 10:30, Adolf Belka <adolf.belka(a)ipfire.org> 
>>>>>>>>> wrote:
>>>>>>>>> Hi All,
>>>>>>>>> I did an update run with autoconf taking it from 2.71 to 2.72
>>>>>>>>> autoconf built without any problems, however collectd then 
>>>>>>>>> failed for a syntax error in the configure file.
>>>>>>>>> Confirmed that this was due to autoconf by changing the 
>>>>>>>>> versions back and forward and the problem went away and then 
>>>>>>>>> came back again.
>>>>>>>>> The collectd build error message is
>>>>>>>>> checking for size_t... yes
>>>>>>>>> checking for uid_t... yes
>>>>>>>>> checking for gid_t... yes
>>>>>>>>> ./configure: line 18710: syntax error near unexpected token 
>>>>>>>>> `newline'
>>>>>>>>> ./configure: line 18710: `yes:'
>>>>>>>>> make: *** [collectd:120: /usr/src/log/collectd-4.10.9] Error 2
>>>>>>>>> The section around line 18710 in the configure file is
>>>>>>>>> 18707    do :
>>>>>>>>> 18708       as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | 
>>>>>>>>> $as_tr_sh`
>>>>>>>>> 18709    ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" 
>>>>>>>>> "$as_ac_Header" "$ac_includes_default"
>>>>>>>>> 18710    if eval test \"x\$"$as_ac_Header"\" = x"yes"; then :
>>>>>>>> There is an extra : here which is probably what causes the 
>>>>>>>> syntax error.
>>>>>>>>> 18711      cat >>confdefs.h <<_ACEOF
>>>>>>>>> 18712    #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1
>>>>>>>>> 18713    _ACEOF
>>>>>>>>> 18714
>>>>>>>>> 18715    else
>>>>>>>>> 18716      with_libiptc="no (header file missing)"
>>>>>>>>> 18717    fi
>>>>>>>>> However there are 15 occurrences of the exact same text as 
>>>>>>>>> line 18710 in the configure file so I am not convinced if that 
>>>>>>>>> line is the root cause for the syntax error but some other 
>>>>>>>>> earlier error that causes a knock-on effect.
>>>>>>>>> However, my knowledge of the coding syntax is definitely not 
>>>>>>>>> enough to figure out what needs to be fixed/changed.
>>>>>>>>> As collectd is a very old version it is likely that some 
>>>>>>>>> structural coding or syntax was fine in the past but now with 
>>>>>>>>> the change from autoconf-2.71 to 2.72 it is no longer allowed, 
>>>>>>>>> or is flagged up when in the past it was just ignored.
>>>>>>>>> There are quite a few changes in autoconf-2.72 with some being 
>>>>>>>>> marked as backwards compatibilities. All of them except one 
>>>>>>>>> say that existing configure scripts will continue working. The 
>>>>>>>>> one that doesn't mention that is the following change:-
>>>>>>>>>     Configure scripts no longer support pre-1989 C compilers.
>>>>>>>>>     Specifically, compilers that *only* implement the original 
>>>>>>>>> “K&R”
>>>>>>>>>     function definition syntax, and not the newer “prototyped” 
>>>>>>>>> syntax,
>>>>>>>>>     will not be able to parse the test programs now emitted by
>>>>>>>>>     AC_CHECK_FUNC, AC_LANG_CALL, and similar macros. 
>>>>>>>>> AC_PROG_CC still
>>>>>>>>>     accepts such compilers, but this may change in the near 
>>>>>>>>> future.
>>>>>>>>>     This change was necessary in order to support the upcoming 
>>>>>>>>> 2024
>>>>>>>>>     edition of the C standard (often referred to as “C23”), 
>>>>>>>>> which will
>>>>>>>>>     officially remove the function declaration syntax used by
>>>>>>>>>     AC_CHECK_FUNC in Autoconf 2.71 and earlier. We feel that 
>>>>>>>>> support
>>>>>>>>>     for compilers that support only C 2024 is more useful, 
>>>>>>>>> nowadays,
>>>>>>>>>     than support for compilers that don’t implement a core 
>>>>>>>>> feature of
>>>>>>>>>     C 1989.
>>>>>>>>> However I am unable to figure out from this if the problem I 
>>>>>>>>> am experiencing is related to this or not. I would not have 
>>>>>>>>> thought so as I don't believe we are using a pre-1989 C compiler.
>>>>>>>>> Any ideas from anyone on how to fix this issue?
>>>>>>>>> There is nothing critical from a security or other 
>>>>>>>>> vulnerability aspect in autoconf-2.72 but it would be nice to 
>>>>>>>>> figure this out before we get to a stage where it has to be 
>>>>>>>>> made to work.
>>>>>>>>> Regards,
>>>>>>>>> Adolf.
>>>>
>>>
>>

-- 
Sent from my laptop