From mboxrd@z Thu Jan 1 00:00:00 1970 From: Adolf Belka To: development@lists.ipfire.org Subject: Re: Failure building collectd when autoconf has been updated to 2.72 Date: Tue, 03 Dec 2024 19:43:36 +0100 Message-ID: In-Reply-To: <78F4DD61-5D41-4F2B-809D-E906505317F4@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6479603592229565845==" List-Id: --===============6479603592229565845== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi Michael, On 03/12/2024 18:44, Michael Tremer wrote: > Okay, this sounds really good then and seems to be a lot easier than expect= ed. >=20 > Are there any changes we would like to make to the graphs in this step? Rem= ove? Add new things? It does seem much easier than expected. Based on that, my preference=20 would be to just do the collectd version change and make sure that=20 everything is really as easy as I have thought. Then we can look at any=20 further changes or additions to graphs at that point. Regards, Adolf. >=20 >> On 29 Nov 2024, at 19:24, Adolf Belka wrote: >> >> Hi Michael, >> >> >> On 25/11/2024 19:25, Adolf Belka wrote: >>> Hi Michael, >>> >>> On 23/11/2024 13:09, Adolf Belka wrote: >>>> Hi Michael, >>>> >>>> On 23/11/2024 10:12, Michael Tremer wrote: >>>>> Hello Adolf, >>>>> >>>>> Hmm, then let us maybe look at migrating collectd. It will be a pain, b= ut it seems that fixing the autoconf problem won=E2=80=99t be much easier. >>>>> >>>>> There is a migration script here: >>>>> >>>>> https://github.com/collectd/collectd/blob/db4a13bdf37b1103c201426145832= 3e26b034eec/contrib/migrate-4-5.px#L107-L108 >>>>> >>>>> Should we maybe start with building a new branch with the latest versio= n of collectd 5 and simply check if a newly installed system is still working= ? I don=E2=80=99t think there will be that many changes required. Would you b= e up for giving that a go? >>>> >>>> I will give that a go and see what happens and we can work from there. >>>> >>> I gave it a go and have been very successful. >>> >>> Building the collectd-5.12.0 did result in various build failures. I thin= k I had to do around 7 build attempts but I progressively moved forward and e= ventually succeeded in a successful build of the new collectd and then the re= st of the IPFire build. >>> >>> I then installed the CU190 iso with collectd-5.12.0 into a vm system. The= fresh install already updates all the rrd files with the updated changes fro= m 4.x to 5.x >>> >>> There were a range of graphs that failed due to, for example, a DS named = ping not being found in the ping rrd file. >>> >>> Checking the ping rrd file with rrdtool info I found that in 5.x the ping= rrd file DS is now "value" in place of "ping". >>> >>> I then found the DEF statement in graphs.pl for the ping graph and change= d the rrd:ping section to rrd:value and the graph started operating and colle= cting data. I also did it with the conntrack entry which changed from "entrop= y" to "value" and again the graph started working. >>> >>> So I know now what I need to do - find all the places where the DS has ch= anged in an rrd file and correct those in the IPFire code. >>> >>> I tested out the migrate-4-5.px file and in its first phase it creates a = script which will update the DS values to their new versions. I can use the o= utput of that script to identify which files have got a change in them, find = where in the code that file is accessed and modify it. >>> >>> We will also need to use the migrate script in the restore portion of the= backup process as the rrd files are in the backup and restoring them will gi= ve files with a DS of "ping" for the ping rrd one, which will not be recognis= ed with the modified DS values in graphs.pl etc. >>> >>> Running the migrate script after doing a restore will convert all the DS = values to their 5.x values. If the entries in the backup are already updated = then the rrdtool tune command run by the migrate script will give the error m= essage >>> >>> ERROR: unknown data source name 'ping' >>> >>> in the example of the ping rrd file but in that case the entry is left as= it is so the migrate script should be able to work on restores with the 4.x = and 5.x rrd values. >>> >>> It looks like both DS names and DS types have been changed for some of th= e plugins. >>> >>> >>> Anyway, things are looking very promising. Should be able to look at test= ing this out in unstable early in the new year. >>> >> So it turned out to be much easier than any of us thought. The changes nee= ded in graphs.pl are only 4 lines to be changed. >> The interface directory becomes the interface-$interface directory and the= re are two lines for that and then the rrd:ping (gateway graph) and rrd:entro= py (conntrack graph) entries need to both be changed to rrd:value. >> >> I also added the migrate-4-5 script into backup.pl >> >> Tested all above out on my vm testbed. Fresh install of IPFire gave the ne= w names and directories as required and everything worked. I also then restor= ed an older backup that had the old names etc and the code in backup.pl also = worked without any hitch. The graphs all worked. >> >> The only thing I haven't done yet is adding the migrate code I put into ba= ckup.pl into the update.sh script. I will do that when next is updated to bec= ome CU191. >> >> I will also just confirm that with the new collectd is happy to build with= autoconf-2.72. Everything I have done so far is with the existing autoconf-2= .71. >> >> I think that the collectd update should be good to go into Core Update 192= early in the new year. >> >> Regards, >> >> Adolf. >> >>> >>> Regards, >>> >>> Adolf. >>> >>>> Regards, >>>> >>>> Adolf >>>> >>>>> >>>>> If we can then run the migration script on existing systems we might al= most be there=E2=80=A6 >>>>> >>>>> -Michael >>>>> >>>>>> On 22 Nov 2024, at 18:01, Adolf Belka wrote: >>>>>> >>>>>> Hi Michael, >>>>>> >>>>>> On 20/11/2024 19:25, Michael Tremer wrote: >>>>>>> Let=E2=80=99s maybe try to avoid running autoconf altogether. >>>>>>> There might be a small chance because only two patches change configu= re.ac. There are others that change Makefile.in, so this might not work at al= l. Let=E2=80=99s see=E2=80=A6 >>>>>>> Could you please remove the patches in src/patches/collectd with numb= er 12 and 13? Then remove the autoreconf commands from lfs/collectd. >>>>>>> Things should hopefully still apply and after the first run of config= ure, make should not try to run autoconf again. Not sure but worth a try. >>>>>> >>>>>> I removed the two patches and also removed the two autoreconf commands= , leaving the autoupdate command in place. >>>>>> >>>>>> Unfortunately it failed to build and had the following error message >>>>>> >>>>>> aclocal.m4:16: warning: this file was generated for autoconf 2.67. >>>>>> You have another version of autoconf. It may work, but is not guarant= eed to. >>>>>> If you have problems, you may need to regenerate the build system enti= rely. >>>>>> To do so, use the procedure documented by the package, typically `auto= reconf'. >>>>>> configure.in:533: error: AC_PROG_CC cannot be called after AM_PROG_CC_= C_O >>>>>> configure.in:533: the top level >>>>>> autom4te: error: /tools_x86_64/bin/m4 failed with exit status: 1 >>>>>> make[1]: *** [Makefile:375: configure] Error 1 >>>>>> make[1]: Leaving directory '/usr/src/collectd-4.10.9' >>>>>> make: *** [collectd:117: /usr/src/log/collectd-4.10.9] Error 2 >>>>>> >>>>>>> I can=E2=80=99t remember what has stopped us from moving with upstrea= m apart from the RRD file format. If this all doesn=E2=80=99t work, maybe we = should look at this all again and see if we can=E2=80=99t migrate to the late= st upstream version again. >>>>>> >>>>>> As far as I can remember it was mentioned that the issue was the chang= ed format and RRD being used in so many places making the update require chan= ges in so many places in IPFire-2.x >>>>>> >>>>>> Regards, >>>>>> >>>>>> Adolf. >>>>>> >>>>>>> Best, >>>>>>> -Michael >>>>>>>> On 19 Nov 2024, at 13:55, Adolf Belka wro= te: >>>>>>>> =EF=BB=BFHi Michael, >>>>>>>> >>>>>>>> Also reran the collectd autoreconf with the autoconf-2.71 and in tha= t case the line has yes:) so something about the autoconf-2.72 is causing tha= t one yes: in the configure file to be missing the ) . There are something li= ke 20 of those yes: entries and with the autoconf-2.72 only the one I show be= low is the one with a missing ), all the rest have a yes:) entry. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Adolf. >>>>>>>> >>>>>>>>> On 19/11/2024 14:43, Adolf Belka wrote: >>>>>>>>> Hi Michael, >>>>>>>>> >>>>>>>>> I obtained the configure created by the autoreconf etc and I have f= ound the error. Here is the affected line 18710 >>>>>>>>> >>>>>>>>> >>>>>>>>> 18706 if test "x$ac_ct_CC" =3D x; then >>>>>>>>> 18707 CC=3D"" >>>>>>>>> 18708 else >>>>>>>>> 18709 case $cross_compiling:$ac_tool_warned in >>>>>>>>> 18710 yes: >>>>>>>>> 18711 >>>>>>>>> 18712 { printf "%s\n" "$as_me:${as_lineno-$LINENO}: WARNING: using = cross tools not prefixed with host triplet" >&5 >>>>>>>>> 18713 printf "%s\n" "$as_me: WARNING: using cross tools not prefixe= d with host triplet" >&2;} >>>>>>>>> 18714 ac_tool_warned=3Dyes ;; >>>>>>>>> 18715 esac >>>>>>>>> 18716 CC=3D$ac_ct_CC >>>>>>>>> 18717 fi >>>>>>>>> 18718 else >>>>>>>>> 18719 CC=3D"$ac_cv_prog_CC" >>>>>>>>> 18720 fi >>>>>>>>> >>>>>>>>> That line should be yes:) so it is missing a right bracket. Not sur= e if you can figure where and why that is occurring but presumably a patch ca= n be written to be applied after the autoreconf etc and before the ./configur= e command >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Adolf. >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 19/11/2024 12:41, Michael Tremer wrote: >>>>>>>>>> Hello Adolf, >>>>>>>>>> In the build script we are running autoreconf which will regenerat= e the configure script: >>>>>>>>>> https://git.ipfire.org/?p=3Dipfire-2.x.git;a=3Dblob;f=3Dlfs/collec= td;h=3Dd1d4ea721386803c31599315de956373417c2dcf;hb=3DHEAD#l119 >>>>>>>>>> Is there any output of that command? There should be some warnings= which might help us to find out what we need to change. >>>>>>>>>>> On 13 Nov 2024, at 10:30, Adolf Belka = wrote: >>>>>>>>>>> Hi All, >>>>>>>>>>> I did an update run with autoconf taking it from 2.71 to 2.72 >>>>>>>>>>> autoconf built without any problems, however collectd then failed= for a syntax error in the configure file. >>>>>>>>>>> Confirmed that this was due to autoconf by changing the versions = back and forward and the problem went away and then came back again. >>>>>>>>>>> The collectd build error message is >>>>>>>>>>> checking for size_t... yes >>>>>>>>>>> checking for uid_t... yes >>>>>>>>>>> checking for gid_t... yes >>>>>>>>>>> ./configure: line 18710: syntax error near unexpected token `newl= ine' >>>>>>>>>>> ./configure: line 18710: `yes:' >>>>>>>>>>> make: *** [collectd:120: /usr/src/log/collectd-4.10.9] Error 2 >>>>>>>>>>> The section around line 18710 in the configure file is >>>>>>>>>>> 18707 do : >>>>>>>>>>> 18708 as_ac_Header=3D`$as_echo "ac_cv_header_$ac_header" | = $as_tr_sh` >>>>>>>>>>> 18709 ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as= _ac_Header" "$ac_includes_default" >>>>>>>>>>> 18710 if eval test \"x\$"$as_ac_Header"\" =3D x"yes"; then : >>>>>>>>>> There is an extra : here which is probably what causes the syntax = error. >>>>>>>>>>> 18711 cat >>confdefs.h <<_ACEOF >>>>>>>>>>> 18712 #define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 >>>>>>>>>>> 18713 _ACEOF >>>>>>>>>>> 18714 >>>>>>>>>>> 18715 else >>>>>>>>>>> 18716 with_libiptc=3D"no (header file missing)" >>>>>>>>>>> 18717 fi >>>>>>>>>>> However there are 15 occurrences of the exact same text as line 1= 8710 in the configure file so I am not convinced if that line is the root cau= se for the syntax error but some other earlier error that causes a knock-on e= ffect. >>>>>>>>>>> However, my knowledge of the coding syntax is definitely not enou= gh to figure out what needs to be fixed/changed. >>>>>>>>>>> As collectd is a very old version it is likely that some structur= al coding or syntax was fine in the past but now with the change from autocon= f-2.71 to 2.72 it is no longer allowed, or is flagged up when in the past it = was just ignored. >>>>>>>>>>> There are quite a few changes in autoconf-2.72 with some being ma= rked as backwards compatibilities. All of them except one say that existing c= onfigure scripts will continue working. The one that doesn't mention that is = the following change:- >>>>>>>>>>> Configure scripts no longer support pre-1989 C compilers. >>>>>>>>>>> Specifically, compilers that *only* implement the original = =E2=80=9CK&R=E2=80=9D >>>>>>>>>>> function definition syntax, and not the newer =E2=80=9Cproto= typed=E2=80=9D syntax, >>>>>>>>>>> will not be able to parse the test programs now emitted by >>>>>>>>>>> AC_CHECK_FUNC, AC_LANG_CALL, and similar macros. AC_PROG_CC = still >>>>>>>>>>> accepts such compilers, but this may change in the near futu= re. >>>>>>>>>>> This change was necessary in order to support the upcoming 2= 024 >>>>>>>>>>> edition of the C standard (often referred to as =E2=80=9CC23= =E2=80=9D), which will >>>>>>>>>>> officially remove the function declaration syntax used by >>>>>>>>>>> AC_CHECK_FUNC in Autoconf 2.71 and earlier. We feel that sup= port >>>>>>>>>>> for compilers that support only C 2024 is more useful, nowad= ays, >>>>>>>>>>> than support for compilers that don=E2=80=99t implement a co= re feature of >>>>>>>>>>> C 1989. >>>>>>>>>>> However I am unable to figure out from this if the problem I am e= xperiencing is related to this or not. I would not have thought so as I don't= believe we are using a pre-1989 C compiler. >>>>>>>>>>> Any ideas from anyone on how to fix this issue? >>>>>>>>>>> There is nothing critical from a security or other vulnerability = aspect in autoconf-2.72 but it would be nice to figure this out before we get= to a stage where it has to be made to work. >>>>>>>>>>> Regards, >>>>>>>>>>> Adolf. >>>>>> >>>>> >>>> >> >> --=20 >> Sent from my laptop >=20 >=20 --=20 Sent from my laptop --===============6479603592229565845==--