From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Failure building collectd when autoconf has been updated to 2.72 Date: Fri, 29 Nov 2024 20:24:46 +0100 Message-ID: <6a427210-9dfc-4a3b-aa1a-918882d68cad@ipfire.org> In-Reply-To: <607a4e76-11ec-451d-b640-15805427f6d3@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============2727204338548192471==" List-Id: --===============2727204338548192471== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael, On 25/11/2024 19:25, Adolf Belka wrote: > Hi Michael, > > On 23/11/2024 13:09, Adolf Belka wrote: >> Hi Michael, >> >> On 23/11/2024 10:12, Michael Tremer wrote: >>> Hello Adolf, >>> >>> Hmm, then let us maybe look at migrating collectd. It will be a=20 >>> pain, but it seems that fixing the autoconf problem won=E2=80=99t be much= =20 >>> easier. >>> >>> There is a migration script here: >>> >>> https://github.com/collectd/collectd/blob/db4a13bdf37b1103c2014261458323e= 26b034eec/contrib/migrate-4-5.px#L107-L108=20 >>> >>> >>> Should we maybe start with building a new branch with the latest=20 >>> version of collectd 5 and simply check if a newly installed system=20 >>> is still working? I don=E2=80=99t think there will be that many changes=20 >>> required. Would you be up for giving that a go? >> >> I will give that a go and see what happens and we can work from there. >> > I gave it a go and have been very successful. > > Building the collectd-5.12.0 did result in various build failures. I=20 > think I had to do around 7 build attempts but I progressively moved=20 > forward and eventually succeeded in a successful build of the new=20 > collectd and then the rest of the IPFire build. > > I then installed the CU190 iso with collectd-5.12.0 into a vm system.=20 > The fresh install already updates all the rrd files with the updated=20 > changes from 4.x to 5.x > > There were a range of graphs that failed due to, for example, a DS=20 > named ping not being found in the ping rrd file. > > Checking the ping rrd file with rrdtool info I found that in 5.x the=20 > ping rrd file DS is now "value" in place of "ping". > > I then found the DEF statement in graphs.pl for the ping graph and=20 > changed the rrd:ping section to rrd:value and the graph started=20 > operating and collecting data. I also did it with the conntrack entry=20 > which changed from "entropy" to "value" and again the graph started=20 > working. > > So I know now what I need to do - find all the places where the DS has=20 > changed in an rrd file and correct those in the IPFire code. > > I tested out the migrate-4-5.px file and in its first phase it creates=20 > a script which will update the DS values to their new versions. I can=20 > use the output of that script to identify which files have got a=20 > change in them, find where in the code that file is accessed and=20 > modify it. > > We will also need to use the migrate script in the restore portion of=20 > the backup process as the rrd files are in the backup and restoring=20 > them will give files with a DS of "ping" for the ping rrd one, which=20 > will not be recognised with the modified DS values in graphs.pl etc. > > Running the migrate script after doing a restore will convert all the=20 > DS values to their 5.x values. If the entries in the backup are=20 > already updated then the rrdtool tune command run by the migrate=20 > script will give the error message > > ERROR: unknown data source name 'ping' > > in the example of the ping rrd file but in that case the entry is left=20 > as it is so the migrate script should be able to work on restores with=20 > the 4.x and 5.x rrd values. > > It looks like both DS names and DS types have been changed for some of=20 > the plugins. > > > Anyway, things are looking very promising. Should be able to look at=20 > testing this out in unstable early in the new year. > So it turned out to be much easier than any of us thought. The changes=20 needed in graphs.pl are only 4 lines to be changed. The interface directory becomes the interface-$interface directory and=20 there are two lines for that and then the rrd:ping (gateway graph) and=20 rrd:entropy (conntrack graph) entries need to both be changed to rrd:value. I also added the migrate-4-5 script into backup.pl Tested all above out on my vm testbed. Fresh install of IPFire gave the=20 new names and directories as required and everything worked. I also then=20 restored an older backup that had the old names etc and the code in=20 backup.pl also worked without any hitch. The graphs all worked. The only thing I haven't done yet is adding the migrate code I put into=20 backup.pl into the update.sh script. I will do that when next is updated=20 to become CU191. I will also just confirm that with the new collectd is happy to build=20 with autoconf-2.72. Everything I have done so far is with the existing=20 autoconf-2.71. I think that the collectd update should be good to go into Core Update=20 192 early in the new year. Regards, Adolf. > > Regards, > > Adolf. > >> Regards, >> >> Adolf >> >>> >>> If we can then run the migration script on existing systems we might=20 >>> almost be there=E2=80=A6 >>> >>> -Michael >>> >>>> On 22 Nov 2024, at 18:01, Adolf Belka wrote: >>>> >>>> Hi Michael, >>>> >>>> On 20/11/2024 19:25, Michael Tremer wrote: >>>>> Let=E2=80=99s maybe try to avoid running autoconf altogether. >>>>> There might be a small chance because only two patches change=20 >>>>> configure.ac. There are others that change Makefile.in, so this=20 >>>>> might not work at all. Let=E2=80=99s see=E2=80=A6 >>>>> Could you please remove the patches in src/patches/collectd with=20 >>>>> number 12 and 13? Then remove the autoreconf commands from=20 >>>>> lfs/collectd. >>>>> Things should hopefully still apply and after the first run of=20 >>>>> configure, make should not try to run autoconf again. Not sure but=20 >>>>> worth a try. >>>> >>>> I removed the two patches and also removed the two autoreconf=20 >>>> commands, leaving the autoupdate command in place. >>>> >>>> Unfortunately it failed to build and had the following error message >>>> >>>> aclocal.m4:16: warning: this file was generated for autoconf 2.67. >>>> You have another version of autoconf.=C2=A0 It may work, but is not=20 >>>> guaranteed to. >>>> If you have problems, you may need to regenerate the build system=20 >>>> entirely. >>>> To do so, use the procedure documented by the package, typically=20 >>>> `autoreconf'. >>>> configure.in:533: error: AC_PROG_CC cannot be called after=20 >>>> AM_PROG_CC_C_O >>>> configure.in:533: the top level >>>> autom4te: error: /tools_x86_64/bin/m4 failed with exit status: 1 >>>> make[1]: *** [Makefile:375: configure] Error 1 >>>> make[1]: Leaving directory '/usr/src/collectd-4.10.9' >>>> make: *** [collectd:117: /usr/src/log/collectd-4.10.9] Error 2 >>>> >>>>> I can=E2=80=99t remember what has stopped us from moving with upstream = >>>>> apart from the RRD file format. If this all doesn=E2=80=99t work, maybe= we=20 >>>>> should look at this all again and see if we can=E2=80=99t migrate to th= e=20 >>>>> latest upstream version again. >>>> >>>> As far as I can remember it was mentioned that the issue was the=20 >>>> changed format and RRD being used in so many places making the=20 >>>> update require changes in so many places in IPFire-2.x >>>> >>>> Regards, >>>> >>>> Adolf. >>>> >>>>> Best, >>>>> -Michael >>>>>> On 19 Nov 2024, at 13:55, Adolf Belka =20 >>>>>> wrote: >>>>>> =EF=BB=BFHi Michael, >>>>>> >>>>>> Also reran the collectd autoreconf with the autoconf-2.71 and in=20 >>>>>> that case the line has yes:) so something about the autoconf-2.72=20 >>>>>> is causing that one yes: in the configure file to be missing the=20 >>>>>> ) . There are something like 20 of those yes: entries and with=20 >>>>>> the autoconf-2.72 only the one I show below is the one with a=20 >>>>>> missing ), all the rest have a yes:) entry. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Adolf. >>>>>> >>>>>>> On 19/11/2024 14:43, Adolf Belka wrote: >>>>>>> Hi Michael, >>>>>>> >>>>>>> I obtained the configure created by the autoreconf etc and I=20 >>>>>>> have found the error. Here is the affected line 18710 >>>>>>> >>>>>>> >>>>>>> 18706=C2=A0=C2=A0 if test "x$ac_ct_CC" =3D x; then >>>>>>> 18707=C2=A0=C2=A0=C2=A0=C2=A0 CC=3D"" >>>>>>> 18708=C2=A0=C2=A0 else >>>>>>> 18709=C2=A0=C2=A0=C2=A0=C2=A0 case $cross_compiling:$ac_tool_warned in >>>>>>> 18710 yes: >>>>>>> 18711 >>>>>>> 18712 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: WARNING:=20 >>>>>>> using cross tools not prefixed with host triplet" >&5 >>>>>>> 18713 printf "%s\n" "$as_me: WARNING: using cross tools not=20 >>>>>>> prefixed with host triplet" >&2;} >>>>>>> 18714 ac_tool_warned=3Dyes ;; >>>>>>> 18715 esac >>>>>>> 18716=C2=A0=C2=A0=C2=A0=C2=A0 CC=3D$ac_ct_CC >>>>>>> 18717=C2=A0=C2=A0 fi >>>>>>> 18718 else >>>>>>> 18719=C2=A0=C2=A0 CC=3D"$ac_cv_prog_CC" >>>>>>> 18720 fi >>>>>>> >>>>>>> That line should be yes:) so it is missing a right bracket. Not=20 >>>>>>> sure if you can figure where and why that is occurring but=20 >>>>>>> presumably a patch can be written to be applied after the=20 >>>>>>> autoreconf etc and before the ./configure command >>>>>>> >>>>>>> Regards, >>>>>>> Adolf. >>>>>>> >>>>>>> >>>>>>>> On 19/11/2024 12:41, Michael Tremer wrote: >>>>>>>> Hello Adolf, >>>>>>>> In the build script we are running autoreconf which will=20 >>>>>>>> regenerate the configure script: >>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dlfs/collectd= ;h=3Dd1d4ea721386803c31599315de956373417c2dcf;hb=3DHEAD#l119=20 >>>>>>>> >>>>>>>> Is there any output of that command? There should be some=20 >>>>>>>> warnings which might help us to find out what we need to change. >>>>>>>>> On 13 Nov 2024, at 10:30, Adolf Belka =20 >>>>>>>>> wrote: >>>>>>>>> Hi All, >>>>>>>>> I did an update run with autoconf taking it from 2.71 to 2.72 >>>>>>>>> autoconf built without any problems, however collectd then=20 >>>>>>>>> failed for a syntax error in the configure file. >>>>>>>>> Confirmed that this was due to autoconf by changing the=20 >>>>>>>>> versions back and forward and the problem went away and then=20 >>>>>>>>> came back again. >>>>>>>>> The collectd build error message is >>>>>>>>> checking for size_t... yes >>>>>>>>> checking for uid_t... yes >>>>>>>>> checking for gid_t... yes >>>>>>>>> ./configure: line 18710: syntax error near unexpected token=20 >>>>>>>>> `newline' >>>>>>>>> ./configure: line 18710: `yes:' >>>>>>>>> make: *** [collectd:120: /usr/src/log/collectd-4.10.9] Error 2 >>>>>>>>> The section around line 18710 in the configure file is >>>>>>>>> 18707=C2=A0=C2=A0=C2=A0 do : >>>>>>>>> 18708=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 as_ac_Header=3D`$as_echo = "ac_cv_header_$ac_header" |=20 >>>>>>>>> $as_tr_sh` >>>>>>>>> 18709=C2=A0=C2=A0=C2=A0 ac_fn_c_check_header_mongrel "$LINENO" "$ac= _header"=20 >>>>>>>>> "$as_ac_Header" "$ac_includes_default" >>>>>>>>> 18710=C2=A0=C2=A0=C2=A0 if eval test \"x\$"$as_ac_Header"\" =3D x"y= es"; then : >>>>>>>> There is an extra : here which is probably what causes the=20 >>>>>>>> syntax error. >>>>>>>>> 18711=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cat >>confdefs.h <<_ACEOF >>>>>>>>> 18712=C2=A0=C2=A0=C2=A0 #define `$as_echo "HAVE_$ac_header" | $as_t= r_cpp` 1 >>>>>>>>> 18713=C2=A0=C2=A0=C2=A0 _ACEOF >>>>>>>>> 18714 >>>>>>>>> 18715=C2=A0=C2=A0=C2=A0 else >>>>>>>>> 18716=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 with_libiptc=3D"no (header file= missing)" >>>>>>>>> 18717=C2=A0=C2=A0=C2=A0 fi >>>>>>>>> However there are 15 occurrences of the exact same text as=20 >>>>>>>>> line 18710 in the configure file so I am not convinced if that=20 >>>>>>>>> line is the root cause for the syntax error but some other=20 >>>>>>>>> earlier error that causes a knock-on effect. >>>>>>>>> However, my knowledge of the coding syntax is definitely not=20 >>>>>>>>> enough to figure out what needs to be fixed/changed. >>>>>>>>> As collectd is a very old version it is likely that some=20 >>>>>>>>> structural coding or syntax was fine in the past but now with=20 >>>>>>>>> the change from autoconf-2.71 to 2.72 it is no longer allowed,=20 >>>>>>>>> or is flagged up when in the past it was just ignored. >>>>>>>>> There are quite a few changes in autoconf-2.72 with some being=20 >>>>>>>>> marked as backwards compatibilities. All of them except one=20 >>>>>>>>> say that existing configure scripts will continue working. The=20 >>>>>>>>> one that doesn't mention that is the following change:- >>>>>>>>> =C2=A0=C2=A0=C2=A0 Configure scripts no longer support pre-1989 C c= ompilers. >>>>>>>>> =C2=A0=C2=A0=C2=A0 Specifically, compilers that *only* implement th= e original=20 >>>>>>>>> =E2=80=9CK&R=E2=80=9D >>>>>>>>> =C2=A0=C2=A0=C2=A0 function definition syntax, and not the newer = =E2=80=9Cprototyped=E2=80=9D=20 >>>>>>>>> syntax, >>>>>>>>> =C2=A0=C2=A0=C2=A0 will not be able to parse the test programs now = emitted by >>>>>>>>> =C2=A0=C2=A0=C2=A0 AC_CHECK_FUNC, AC_LANG_CALL, and similar macros.= =20 >>>>>>>>> AC_PROG_CC still >>>>>>>>> =C2=A0=C2=A0=C2=A0 accepts such compilers, but this may change in t= he near=20 >>>>>>>>> future. >>>>>>>>> =C2=A0=C2=A0=C2=A0 This change was necessary in order to support th= e upcoming=20 >>>>>>>>> 2024 >>>>>>>>> =C2=A0=C2=A0=C2=A0 edition of the C standard (often referred to as = =E2=80=9CC23=E2=80=9D),=20 >>>>>>>>> which will >>>>>>>>> =C2=A0=C2=A0=C2=A0 officially remove the function declaration synta= x used by >>>>>>>>> =C2=A0=C2=A0=C2=A0 AC_CHECK_FUNC in Autoconf 2.71 and earlier. We f= eel that=20 >>>>>>>>> support >>>>>>>>> =C2=A0=C2=A0=C2=A0 for compilers that support only C 2024 is more u= seful,=20 >>>>>>>>> nowadays, >>>>>>>>> =C2=A0=C2=A0=C2=A0 than support for compilers that don=E2=80=99t im= plement a core=20 >>>>>>>>> feature of >>>>>>>>> =C2=A0=C2=A0=C2=A0 C 1989. >>>>>>>>> However I am unable to figure out from this if the problem I=20 >>>>>>>>> am experiencing is related to this or not. I would not have=20 >>>>>>>>> thought so as I don't believe we are using a pre-1989 C compiler. >>>>>>>>> Any ideas from anyone on how to fix this issue? >>>>>>>>> There is nothing critical from a security or other=20 >>>>>>>>> vulnerability aspect in autoconf-2.72 but it would be nice to=20 >>>>>>>>> figure this out before we get to a stage where it has to be=20 >>>>>>>>> made to work. >>>>>>>>> Regards, >>>>>>>>> Adolf. >>>> >>> >> --=20 Sent from my laptop --===============2727204338548192471==--