From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Failure building collectd when autoconf has been updated to 2.72 Date: Wed, 04 Dec 2024 10:50:11 +0000 Message-ID: <196D8D42-02C9-4250-9C8E-08EBAE787A71@ipfire.org> In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0222648733237639724==" List-Id: --===============0222648733237639724== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Okay, just to recap from yesterday=E2=80=99s call: We are going to migrate to collectd 5 first and ship it as soon as possible, = but only whenever it is ready We might want to check how collectd 6 is coming along, but it seems that ther= e is nothing new in it that we would benefit from. Maybe we have some time to tidy up the graphs and maybe there are some new, u= seful plugins. -Michael > On 3 Dec 2024, at 18:43, Adolf Belka wrote: >=20 > Hi Michael, >=20 > On 03/12/2024 18:44, Michael Tremer wrote: >> Okay, this sounds really good then and seems to be a lot easier than expec= ted. >> Are there any changes we would like to make to the graphs in this step? Re= move? Add new things? >=20 > It does seem much easier than expected. Based on that, my preference would = be to just do the collectd version change and make sure that everything is re= ally as easy as I have thought. Then we can look at any further changes or ad= ditions to graphs at that point. >=20 > Regards, >=20 > Adolf. >=20 >>> On 29 Nov 2024, at 19:24, Adolf Belka wrote: >>>=20 >>> Hi Michael, >>>=20 >>>=20 >>> On 25/11/2024 19:25, Adolf Belka wrote: >>>> Hi Michael, >>>>=20 >>>> On 23/11/2024 13:09, Adolf Belka wrote: >>>>> Hi Michael, >>>>>=20 >>>>> On 23/11/2024 10:12, Michael Tremer wrote: >>>>>> Hello Adolf, >>>>>>=20 >>>>>> Hmm, then let us maybe look at migrating collectd. It will be a pain, = but it seems that fixing the autoconf problem won=E2=80=99t be much easier. >>>>>>=20 >>>>>> There is a migration script here: >>>>>>=20 >>>>>> https://github.com/collectd/collectd/blob/db4a13bdf37b1103c20142614583= 23e26b034eec/contrib/migrate-4-5.px#L107-L108 >>>>>>=20 >>>>>> Should we maybe start with building a new branch with the latest versi= on of collectd 5 and simply check if a newly installed system is still workin= g? I don=E2=80=99t think there will be that many changes required. Would you = be up for giving that a go? >>>>>=20 >>>>> I will give that a go and see what happens and we can work from there. >>>>>=20 >>>> I gave it a go and have been very successful. >>>>=20 >>>> Building the collectd-5.12.0 did result in various build failures. I thi= nk I had to do around 7 build attempts but I progressively moved forward and = eventually succeeded in a successful build of the new collectd and then the r= est of the IPFire build. >>>>=20 >>>> I then installed the CU190 iso with collectd-5.12.0 into a vm system. Th= e fresh install already updates all the rrd files with the updated changes fr= om 4.x to 5.x >>>>=20 >>>> There were a range of graphs that failed due to, for example, a DS named= ping not being found in the ping rrd file. >>>>=20 >>>> Checking the ping rrd file with rrdtool info I found that in 5.x the pin= g rrd file DS is now "value" in place of "ping". >>>>=20 >>>> I then found the DEF statement in graphs.pl for the ping graph and chang= ed the rrd:ping section to rrd:value and the graph started operating and coll= ecting data. I also did it with the conntrack entry which changed from "entro= py" to "value" and again the graph started working. >>>>=20 >>>> So I know now what I need to do - find all the places where the DS has c= hanged in an rrd file and correct those in the IPFire code. >>>>=20 >>>> I tested out the migrate-4-5.px file and in its first phase it creates a= script which will update the DS values to their new versions. I can use the = output of that script to identify which files have got a change in them, find= where in the code that file is accessed and modify it. >>>>=20 >>>> We will also need to use the migrate script in the restore portion of th= e backup process as the rrd files are in the backup and restoring them will g= ive files with a DS of "ping" for the ping rrd one, which will not be recogni= sed with the modified DS values in graphs.pl etc. >>>>=20 >>>> Running the migrate script after doing a restore will convert all the DS= values to their 5.x values. If the entries in the backup are already updated= then the rrdtool tune command run by the migrate script will give the error = message >>>>=20 >>>> ERROR: unknown data source name 'ping' >>>>=20 >>>> in the example of the ping rrd file but in that case the entry is left a= s it is so the migrate script should be able to work on restores with the 4.x= and 5.x rrd values. >>>>=20 >>>> It looks like both DS names and DS types have been changed for some of t= he plugins. >>>>=20 >>>>=20 >>>> Anyway, things are looking very promising. Should be able to look at tes= ting this out in unstable early in the new year. >>>>=20 >>> So it turned out to be much easier than any of us thought. The changes ne= eded in graphs.pl are only 4 lines to be changed. >>> The interface directory becomes the interface-$interface directory and th= ere are two lines for that and then the rrd:ping (gateway graph) and rrd:entr= opy (conntrack graph) entries need to both be changed to rrd:value. >>>=20 >>> I also added the migrate-4-5 script into backup.pl >>>=20 >>> Tested all above out on my vm testbed. Fresh install of IPFire gave the n= ew names and directories as required and everything worked. I also then resto= red an older backup that had the old names etc and the code in backup.pl also= worked without any hitch. The graphs all worked. >>>=20 >>> The only thing I haven't done yet is adding the migrate code I put into b= ackup.pl into the update.sh script. I will do that when next is updated to be= come CU191. >>>=20 >>> I will also just confirm that with the new collectd is happy to build wit= h autoconf-2.72. Everything I have done so far is with the existing autoconf-= 2.71. >>>=20 >>> I think that the collectd update should be good to go into Core Update 19= 2 early in the new year. >>>=20 >>> Regards, >>>=20 >>> Adolf. >>>=20 >>>>=20 >>>> Regards, >>>>=20 >>>> Adolf. >>>>=20 >>>>> Regards, >>>>>=20 >>>>> Adolf >>>>>=20 >>>>>>=20 >>>>>> If we can then run the migration script on existing systems we might a= lmost be there=E2=80=A6 >>>>>>=20 >>>>>> -Michael >>>>>>=20 >>>>>>> On 22 Nov 2024, at 18:01, Adolf Belka wrot= e: >>>>>>>=20 >>>>>>> Hi Michael, >>>>>>>=20 >>>>>>> On 20/11/2024 19:25, Michael Tremer wrote: >>>>>>>> Let=E2=80=99s maybe try to avoid running autoconf altogether. >>>>>>>> There might be a small chance because only two patches change config= ure.ac. There are others that change Makefile.in, so this might not work at a= ll. Let=E2=80=99s see=E2=80=A6 >>>>>>>> Could you please remove the patches in src/patches/collectd with num= ber 12 and 13? Then remove the autoreconf commands from lfs/collectd. >>>>>>>> Things should hopefully still apply and after the first run of confi= gure, make should not try to run autoconf again. Not sure but worth a try. >>>>>>>=20 >>>>>>> I removed the two patches and also removed the two autoreconf command= s, leaving the autoupdate command in place. >>>>>>>=20 >>>>>>> Unfortunately it failed to build and had the following error message >>>>>>>=20 >>>>>>> aclocal.m4:16: warning: this file was generated for autoconf 2.67. >>>>>>> You have another version of autoconf. It may work, but is not guaran= teed to. >>>>>>> If you have problems, you may need to regenerate the build system ent= irely. >>>>>>> To do so, use the procedure documented by the package, typically `aut= oreconf'. >>>>>>> configure.in:533: error: AC_PROG_CC cannot be called after AM_PROG_CC= _C_O >>>>>>> configure.in:533: the top level >>>>>>> autom4te: error: /tools_x86_64/bin/m4 failed with exit status: 1 >>>>>>> make[1]: *** [Makefile:375: configure] Error 1 >>>>>>> make[1]: Leaving directory '/usr/src/collectd-4.10.9' >>>>>>> make: *** [collectd:117: /usr/src/log/collectd-4.10.9] Error 2 >>>>>>>=20 >>>>>>>> I can=E2=80=99t remember what has stopped us from moving with upstre= am apart from the RRD file format. If this all doesn=E2=80=99t work, maybe we= should look at this all again and see if we can=E2=80=99t migrate to the lat= est upstream version again. >>>>>>>=20 >>>>>>> As far as I can remember it was mentioned that the issue was the chan= ged format and RRD being used in so many places making the update require cha= nges in so many places in IPFire-2.x >>>>>>>=20 >>>>>>> Regards, >>>>>>>=20 >>>>>>> Adolf. >>>>>>>=20 >>>>>>>> Best, >>>>>>>> -Michael >>>>>>>>> On 19 Nov 2024, at 13:55, Adolf Belka wr= ote: >>>>>>>>> =EF=BB=BFHi Michael, >>>>>>>>>=20 >>>>>>>>> Also reran the collectd autoreconf with the autoconf-2.71 and in th= at case the line has yes:) so something about the autoconf-2.72 is causing th= at one yes: in the configure file to be missing the ) . There are something l= ike 20 of those yes: entries and with the autoconf-2.72 only the one I show b= elow is the one with a missing ), all the rest have a yes:) entry. >>>>>>>>>=20 >>>>>>>>> Regards, >>>>>>>>>=20 >>>>>>>>> Adolf. >>>>>>>>>=20 >>>>>>>>>> On 19/11/2024 14:43, Adolf Belka wrote: >>>>>>>>>> Hi Michael, >>>>>>>>>>=20 >>>>>>>>>> I obtained the configure created by the autoreconf etc and I have = found the error. Here is the affected line 18710 >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>> 18706 if test "x$ac_ct_CC" =3D x; then >>>>>>>>>> 18707 CC=3D"" >>>>>>>>>> 18708 else >>>>>>>>>> 18709 case $cross_compiling:$ac_tool_warned in >>>>>>>>>> 18710 yes: >>>>>>>>>> 18711 >>>>>>>>>> 18712 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: WARNING: using= cross tools not prefixed with host triplet" >&5 >>>>>>>>>> 18713 printf "%s\n" "$as_me: WARNING: using cross tools not prefix= ed with host triplet" >&2;} >>>>>>>>>> 18714 ac_tool_warned=3Dyes ;; >>>>>>>>>> 18715 esac >>>>>>>>>> 18716 CC=3D$ac_ct_CC >>>>>>>>>> 18717 fi >>>>>>>>>> 18718 else >>>>>>>>>> 18719 CC=3D"$ac_cv_prog_CC" >>>>>>>>>> 18720 fi >>>>>>>>>>=20 >>>>>>>>>> That line should be yes:) so it is missing a right bracket. Not su= re if you can figure where and why that is occurring but presumably a patch c= an be written to be applied after the autoreconf etc and before the ./configu= re command >>>>>>>>>>=20 >>>>>>>>>> Regards, >>>>>>>>>> Adolf. >>>>>>>>>>=20 >>>>>>>>>>=20 >>>>>>>>>>> On 19/11/2024 12:41, Michael Tremer wrote: >>>>>>>>>>> Hello Adolf, >>>>>>>>>>> In the build script we are running autoreconf which will regenera= te the configure script: >>>>>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dlfs/colle= ctd;h=3Dd1d4ea721386803c31599315de956373417c2dcf;hb=3DHEAD#l119 >>>>>>>>>>> Is there any output of that command? There should be some warning= s which might help us to find out what we need to change. >>>>>>>>>>>> On 13 Nov 2024, at 10:30, Adolf Belka = wrote: >>>>>>>>>>>> Hi All, >>>>>>>>>>>> I did an update run with autoconf taking it from 2.71 to 2.72 >>>>>>>>>>>> autoconf built without any problems, however collectd then faile= d for a syntax error in the configure file. >>>>>>>>>>>> Confirmed that this was due to autoconf by changing the versions= back and forward and the problem went away and then came back again. >>>>>>>>>>>> The collectd build error message is >>>>>>>>>>>> checking for size_t... yes >>>>>>>>>>>> checking for uid_t... yes >>>>>>>>>>>> checking for gid_t... yes >>>>>>>>>>>> ./configure: line 18710: syntax error near unexpected token `new= line' >>>>>>>>>>>> ./configure: line 18710: `yes:' >>>>>>>>>>>> make: *** [collectd:120: /usr/src/log/collectd-4.10.9] Error 2 >>>>>>>>>>>> The section around line 18710 in the configure file is >>>>>>>>>>>> 18707 do : >>>>>>>>>>>> 18708 as_ac_Header=3D`$as_echo "ac_cv_header_$ac_header" |= $as_tr_sh` >>>>>>>>>>>> 18709 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$a= s_ac_Header" "$ac_includes_default" >>>>>>>>>>>> 18710 if eval test \"x\$"$as_ac_Header"\" =3D x"yes"; then : >>>>>>>>>>> There is an extra : here which is probably what causes the syntax= error. >>>>>>>>>>>> 18711 cat >>confdefs.h <<_ACEOF >>>>>>>>>>>> 18712 #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 >>>>>>>>>>>> 18713 _ACEOF >>>>>>>>>>>> 18714 >>>>>>>>>>>> 18715 else >>>>>>>>>>>> 18716 with_libiptc=3D"no (header file missing)" >>>>>>>>>>>> 18717 fi >>>>>>>>>>>> However there are 15 occurrences of the exact same text as line = 18710 in the configure file so I am not convinced if that line is the root ca= use for the syntax error but some other earlier error that causes a knock-on = effect. >>>>>>>>>>>> However, my knowledge of the coding syntax is definitely not eno= ugh to figure out what needs to be fixed/changed. >>>>>>>>>>>> As collectd is a very old version it is likely that some structu= ral coding or syntax was fine in the past but now with the change from autoco= nf-2.71 to 2.72 it is no longer allowed, or is flagged up when in the past it= was just ignored. >>>>>>>>>>>> There are quite a few changes in autoconf-2.72 with some being m= arked as backwards compatibilities. All of them except one say that existing = configure scripts will continue working. The one that doesn't mention that is= the following change:- >>>>>>>>>>>> Configure scripts no longer support pre-1989 C compilers. >>>>>>>>>>>> Specifically, compilers that *only* implement the original = =E2=80=9CK&R=E2=80=9D >>>>>>>>>>>> function definition syntax, and not the newer =E2=80=9Cproto= typed=E2=80=9D syntax, >>>>>>>>>>>> will not be able to parse the test programs now emitted by >>>>>>>>>>>> AC_CHECK_FUNC, AC_LANG_CALL, and similar macros. AC_PROG_CC = still >>>>>>>>>>>> accepts such compilers, but this may change in the near futu= re. >>>>>>>>>>>> This change was necessary in order to support the upcoming 2= 024 >>>>>>>>>>>> edition of the C standard (often referred to as =E2=80=9CC23= =E2=80=9D), which will >>>>>>>>>>>> officially remove the function declaration syntax used by >>>>>>>>>>>> AC_CHECK_FUNC in Autoconf 2.71 and earlier. We feel that sup= port >>>>>>>>>>>> for compilers that support only C 2024 is more useful, nowad= ays, >>>>>>>>>>>> than support for compilers that don=E2=80=99t implement a co= re feature of >>>>>>>>>>>> C 1989. >>>>>>>>>>>> However I am unable to figure out from this if the problem I am = experiencing is related to this or not. I would not have thought so as I don'= t believe we are using a pre-1989 C compiler. >>>>>>>>>>>> Any ideas from anyone on how to fix this issue? >>>>>>>>>>>> There is nothing critical from a security or other vulnerability= aspect in autoconf-2.72 but it would be nice to figure this out before we ge= t to a stage where it has to be made to work. >>>>>>>>>>>> Regards, >>>>>>>>>>>> Adolf. >>>>>>>=20 >>>>>>=20 >>>>>=20 >>>=20 >>> --=20 >>> Sent from my laptop >=20 > --=20 > Sent from my laptop --===============0222648733237639724==--