From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michael Tremer To: development@lists.ipfire.org Subject: Re: Problem with trying to build qemu-9.0.2 - it requires dtc for all architectures Date: Wed, 04 Sep 2024 21:49:01 +0200 Message-ID: In-Reply-To: <182681ee-7f75-4117-8ded-978757fadf94@ipfire.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1570475366635813482==" List-Id: --===============1570475366635813482== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hello, > On 4 Sep 2024, at 15:37, Adolf Belka wrote: >=20 > Hi Michael, >=20 > On 04/09/2024 11:34, Michael Tremer wrote: >> Hello, >>> On 4 Sep 2024, at 09:22, Adolf Belka wrote: >>>=20 >>> Hi Michael, >>>=20 >>> On 04/09/2024 08:58, Michael Tremer wrote: >>>> Hello, >>>>> On 3 Sep 2024, at 21:06, Adolf Belka wrote: >>>>>=20 >>>>> Hi All, >>>>>=20 >>>>> On 03/09/2024 15:25, Adolf Belka wrote: >>>>>> Hi Michael, >>>>>>=20 >>>>>> On 03/09/2024 09:24, Michael Tremer wrote: >>>>>>> Hello Adolf, >>>>>>>=20 >>>>>>> Yes, you can make those changes. >>>>>>>=20 >>>>>>> dtc is the device tree compiler and it should be quick to build. We j= ust didn=E2=80=99t do that since we didn=E2=80=99t have any use for it apart = from aarch64. >>>>>>>=20 >>>>>>> Move it wherever you want. It should not have any further dependencie= s. We can build it for all architectures. >>>>>>=20 >>>>>> I did that and qemu found libfdt but then said that it was too old and= that a min of 1.5.1 was required but the updated version of dtc I had built = was version 1.7.1 >>>>>>=20 >>>>> I have found the section in meson.build where it checks for the fdt lib= rary and it seems to me that the code says that if the system fdt is being us= ed then say that it is too old. I can't see any actual check of the version n= umber of libfdt. >>>> Hmm, the code is not the cleanest, but it should perform a few checks be= fore it throws that error. The crucial part is here: >>>>> if fdt.found() and cc.links(''=E2=80=99 >>>>> ...''', >>>>> dependencies: fdt) >>>>> fdt_opt =3D 'system' >>>>> elif fdt_opt =3D=3D 'system' >>>>> error('system libfdt requested, but it is too old (1.5.1 or newe= r required)') >>>>> else >>>>> fdt_opt =3D 'internal' >>>>> fdt =3D not_found >>>>> endif >>>> It first of all checks if fdt.found() returns true, it so, it will check= to compile and build the program in cc.links(=E2=80=A6). I assume that the f= unction fdt_find_max_phandle() does not exitt before version 1.5.1 and theref= ore the program should not build with older versions. Then it will move to th= e =E2=80=9Celif fdt_opt =3D=3D =E2=80=99system=E2=80=99=E2=80=9D part and com= plain that the library that is already installed is incompatible. >>>> So the check should work and actually do something. >>>> Whenever a program bundles a library, we should remove it. Bundles libra= ries are evil because lets say there is a problem in libfdt, we will patch th= e package, but we don=E2=80=99t know what other programs bring their own vers= ion which won=E2=80=99t receive the fix. This is particularly bad for anythin= g security-related as parts of the distribution will still be vulnerable. >>>=20 >>> Then when I am doing my package updates if I see any bundled stuff then I= will check how to use a system version instead. >>>> Shared libraries (like libfdt.so .N) are nicer becaus= e they can be pulled in by any program that needs the functionality and can b= e replaced with an improved version without any regard of what programs link = against it - for as long as there are no breaking changes. >>>> Rust for example is going the opposite way where everything is staticall= y linked which is a security nightmare as described above. Every program that= uses a vulnerable crate will have to be rebuilt. But how do you find them al= l? >>>> There is more about bundles libraries here: https://fedoraproject.org/wi= ki/Bundled_Libraries >>>>> Can you have a check through this following code and see if my interpre= tation is correct or if I have just misunderstood how the code is supposed to= be working. >>>>>=20 >>>>> It also looks like the check for the fdt code being internal has not be= en removed from it even though dtc is no longer bundled with qemu. >>>> In that case, we should set fdt=3Dsystem just to make sure that QEMU doe= s the right thing during the build. >>>=20 >>> Okay, I will try that and see what happens. >> For as long as you have built dtc before QEMU this should work fine. >=20 > That change was not enough on its own to make the libfdt detection work. >=20 > I got feedback from Thomas Huth on the qemu gitlab site saying that the err= or message might be a bit misleading and that the issue was more likely that = the libfdt was simply not usable. He said that more info should be in the mes= on-logs/meson-log.txt file >=20 > Commenting out the rm -rf $(DIR_APP) command in the qemu lfs meant that I c= ould read the build/meson-logs/meson-log.txt and that showed that the #includ= e command was not finding the libfdt.h file. If the build aborts, you can always run ./make.sh shell, cd into /usr/src and= you should find the source when it aborted. You don=E2=80=99t need to remove= the rm command, as it will never be reached when the build aborts. > This turned out to be that the dtc build did not have PREFIX=3D/usr and so = those files were being put in /include/ and not /usr/include/ >=20 > So I added PREFIX=3D/usr to the end of the dtc make install line and qemu t= hen successfully built. Should this not be specified with the first build command - i.e. the equivale= nt of ./configure? > I will now do a clean and fresh build to confirm that everything stays work= ing but I might be able to submit a patch for qemu-9.0.2 now. >=20 > Is there any problem with the dtc command being changed from >=20 > cd $(DIR_APP) && make HOME=3D install >=20 > to >=20 > cd $(DIR_APP) && make HOME=3D install PREFIX=3D/usr This is an acceptable change. > in terms of the aarch64 requirements? And aarch64 does not differ from x86 and the others=E2=80=A6 -Michael >=20 > Regards, > Adolf. >=20 >>>> Did I not merge any patches yesterday? Is that version not affected? >>>=20 >>> Yes, you did but that was for version 9.0.0 which still had the bundled d= tc. I thought it was still worth updating as there have been 11 update versio= ns since the 8.2.1 version we currently have in stable. >> Ah, that makes sense. >>> I will also have a look at the other packages they have in the subproject= s directory to see if our build uses any of them. >> Thank you! >> -Michael >>> Regards, >>>=20 >>> Adolf. >>>> -Michael >>>>> fdt =3D not_found >>>>> fdt_opt =3D get_option('fdt') >>>>> if fdt_required.length() > 0 or fdt_opt =3D=3D 'enabled' >>>>> if fdt_opt =3D=3D 'disabled' >>>>> error('fdt disabled but required by targets ' + ', '.join(fdt_requi= red)) >>>>> endif >>>>>=20 >>>>> if fdt_opt in ['enabled', 'auto', 'system'] >>>>> if get_option('wrap_mode') =3D=3D 'nodownload' >>>>> fdt_opt =3D 'system' >>>>> endif >>>>> fdt =3D cc.find_library('fdt', required: fdt_opt =3D=3D 'system') >>>>> if fdt.found() and cc.links(''' >>>>> #include >>>>> #include >>>>> int main(void) { fdt_find_max_phandle(NULL, NULL); return 0; }''= ', >>>>> dependencies: fdt) >>>>> fdt_opt =3D 'system' >>>>> elif fdt_opt =3D=3D 'system' >>>>> error('system libfdt requested, but it is too old (1.5.1 or newe= r required)') >>>>> else >>>>> fdt_opt =3D 'internal' >>>>> fdt =3D not_found >>>>> endif >>>>> endif >>>>> if not fdt.found() >>>>> assert(fdt_opt =3D=3D 'internal') >>>>> libfdt_proj =3D subproject('dtc', required: true, >>>>> default_options: ['tools=3Dfalse', 'yaml= =3Ddisabled', >>>>> 'python=3Ddisabled', 'de= fault_library=3Dstatic']) >>>>> fdt =3D libfdt_proj.get_variable('libfdt_dep') >>>>> endif >>>>> else >>>>> fdt_opt =3D 'disabled' >>>>> endif >>>>>=20 >>>>>=20 >>>>>> It looks like qemu can correctly detect the presence of libfdt but not= the version number. >>>>>>=20 >>>>>> I have raised this as an issue in the qemu gitlab site. >>>>>>=20 >>>>>> Version 9.0.0 still had the dtc subproject included in the source tarb= all, so I am currently building that and have left the IPFire dtc where it wa= s for the moment. >>>>>>=20 >>>>>> This will still update qemu by 11 version numbers. >>>>>>=20 >>>>>> I will wait to see what response I get back from the qemu team for upd= ating to 9.0.2 >>>>>>=20 >>>>>> Regards, >>>>>> Adolf. >>>>>>=20 >>>>>>>=20 >>>>>>> -Michael >>>>>>>=20 >>>>>>>> On 2 Sep 2024, at 19:46, Adolf Belka wrot= e: >>>>>>>>=20 >>>>>>>> Hi All, >>>>>>>>=20 >>>>>>>> I am trying to do an update of the qemu package. >>>>>>>>=20 >>>>>>>> It is failing for a missing fdt library which it says is required by= targets x86_64-softmmu, aarch64-softmmu and riscv64-softmmu. >>>>>>>>=20 >>>>>>>> The previous version that we currently have, 8.1.2 has a bundled dtc= which includes the fdt library. That has been removed, probably with the cha= nge to the 9.x branch as it is not in 9.0.2 >>>>>>>>=20 >>>>>>>> We have the dtc package in make but it is after qemu. >>>>>>>>=20 >>>>>>>> I could move it to before qemu, however I also note that the dtc pac= kage is specified only for aarch64 but if we need to have the above softmmu t= argets specified then qemu is requiring the fdt library for all our architect= ures. >>>>>>>>=20 >>>>>>>> Before I move dtc and change it to be for all architectures, I just = want to flag this up and confirm that it is the right thing to do. >>>>>>>>=20 >>>>>>>> Regards, >>>>>>>>=20 >>>>>>>> Adolf. --===============1570475366635813482==--