Errors at cdrom stage when building the python3 update package

Errors at cdrom stage when building the python3 update package

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

Hi everyone,

I have been working on the python3 update. With python3 updated then some of the individual python packages required to be updated as they failed to build.


To make it simpler I updated all the python3 packages that had new versions and this also resulted in some rust packages needing to be updated. After several rounds then everything built without any problems until it came to the cdrom stage.


I got the following errors and I have no idea what I have done wrong to make this happen. The error messages unfortunately aren't much help to me.

Can anyone help with what the issue is causing the attached log file errors.


Regards,

Adolf.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: _build.packages.log --]
[-- Type: text/x-log, Size: 5051 bytes --]

... see detailed log in _build.*.log files
Oct 15 15:24:15: Building cdrom + cd /usr/src/lfs
+ make -f cdrom LFS_BASEDIR=/usr/src install
# Update /etc/system-release, because the string might have changed
# since stage2 has been executed..
echo "IPFire 2.27 (x86_64) - core171 Development Build: python-3.10.8/a4097795-dirty" > /etc/system-release
# Create a directory to authorise the CDROM in
rm -rf /tmp/cdrom && mkdir -p /tmp/cdrom
# Clear mtab (prevents .journal problems)
rm -vf /etc/mtab
removed '/etc/mtab'
ln -s /proc/self/mounts /etc/mtab
# Create filelist for packaging.
BUILDTARGET="x86_64-pc-linux-gnu" BUILD_ARCH="x86_64" KVER="5.15.71" \
	/usr/src/src/scripts/archive.files \
	/usr/src/config/rootfiles/common \
	> /tmp/ROOTFILES
# Compress root filesystem
# Reason for this tar+untar+tar is removing of entries listed two or more in src/ROOTFILES
rm -rf /tmp/root && mkdir -p /tmp/root
tar --format=pax --acls --xattrs --xattrs-include='*' -c --exclude='#*' --exclude='proc/*' --exclude='dev/pts/*' --exclude='tmp/*' \
	--exclude='__pycache__' \
	-C / --files-from=/tmp/ROOTFILES | tar --format=pax --acls --xattrs --xattrs-include='*' -x -C /tmp/root
tar: /tmp/ROOTFILES:2: unrecognized option
tar: /tmp/ROOTFILES:3: unrecognized option
tar: /tmp/ROOTFILES:4: unrecognized option
tar: /tmp/ROOTFILES:5: unrecognized option
tar: /tmp/ROOTFILES:6: unrecognized option
tar: /tmp/ROOTFILES:7: unrecognized option
tar: /tmp/ROOTFILES:8: unrecognized option
tar: /tmp/ROOTFILES:9: unrecognized option
tar: /tmp/ROOTFILES:10: unrecognized option
tar: /tmp/ROOTFILES:11: unrecognized option
tar: /tmp/ROOTFILES:12: unrecognized option
tar: /tmp/ROOTFILES:13: unrecognized option
tar: /tmp/ROOTFILES:14: unrecognized option
tar: /tmp/ROOTFILES:15: unrecognized option
tar: /tmp/ROOTFILES:16: unrecognized option
tar: /tmp/ROOTFILES:17: unrecognized option
tar: /tmp/ROOTFILES:18: unrecognized option
tar: /tmp/ROOTFILES:19: unrecognized option
tar: /tmp/ROOTFILES:20: unrecognized option
tar: /tmp/ROOTFILES:21: unrecognized option
tar: /tmp/ROOTFILES:22: unrecognized option
tar: /tmp/ROOTFILES:23: unrecognized option
tar: /tmp/ROOTFILES:24: unrecognized option
tar: /tmp/ROOTFILES:25: unrecognized option
tar: /tmp/ROOTFILES:26: unrecognized option
tar: /tmp/ROOTFILES:27: unrecognized option
tar: /tmp/ROOTFILES:28: unrecognized option
tar: /tmp/ROOTFILES:29: unrecognized option
tar: /tmp/ROOTFILES:30: unrecognized option
tar: /tmp/ROOTFILES:31: unrecognized option
tar: /tmp/ROOTFILES:32: unrecognized option
tar: /tmp/ROOTFILES:33: unrecognized option
tar: /tmp/ROOTFILES:34: unrecognized option
tar: /tmp/ROOTFILES:35: unrecognized option
tar: /tmp/ROOTFILES:36: unrecognized option
tar: /tmp/ROOTFILES:37: unrecognized option
tar: /tmp/ROOTFILES:38: unrecognized option
tar: /tmp/ROOTFILES:39: unrecognized option
tar: /tmp/ROOTFILES:40: unrecognized option
tar: /tmp/ROOTFILES:41: unrecognized option
tar: /tmp/ROOTFILES:42: unrecognized option
tar: /tmp/ROOTFILES:43: unrecognized option
tar: /tmp/ROOTFILES:44: unrecognized option
tar: /tmp/ROOTFILES:45: unrecognized option
tar: /tmp/ROOTFILES:46: unrecognized option
tar: /tmp/ROOTFILES:47: unrecognized option
tar: /tmp/ROOTFILES:48: unrecognized option
tar: /tmp/ROOTFILES:49: unrecognized option
tar: /tmp/ROOTFILES:50: unrecognized option
tar: /tmp/ROOTFILES:51: unrecognized option
tar: /tmp/ROOTFILES:52: unrecognized option
tar: /tmp/ROOTFILES:53: unrecognized option
tar: /tmp/ROOTFILES:54: unrecognized option
tar: /tmp/ROOTFILES:55: unrecognized option
tar: /tmp/ROOTFILES:56: unrecognized option
tar: /tmp/ROOTFILES:57: unrecognized option
tar: /tmp/ROOTFILES:58: unrecognized option
tar: /tmp/ROOTFILES:59: unrecognized option
tar: /tmp/ROOTFILES:60: unrecognized option
tar: /tmp/ROOTFILES:61: unrecognized option
tar: /tmp/ROOTFILES:62: unrecognized option
tar: /tmp/ROOTFILES:63: unrecognized option
tar: /tmp/ROOTFILES:64: unrecognized option
tar: /tmp/ROOTFILES:65: unrecognized option
tar: /tmp/ROOTFILES:66: unrecognized option
tar: /tmp/ROOTFILES:67: unrecognized option
tar: /tmp/ROOTFILES:68: unrecognized option
tar: /tmp/ROOTFILES:69: unrecognized option
tar: /tmp/ROOTFILES:70: unrecognized option
tar: /tmp/ROOTFILES:71: unrecognized option
tar: /tmp/ROOTFILES:72: unrecognized option
tar: /tmp/ROOTFILES:73: unrecognized option
tar: /tmp/ROOTFILES:74: unrecognized option
tar: /tmp/ROOTFILES:75: unrecognized option
tar: /tmp/ROOTFILES:76: unrecognized option
tar: /tmp/ROOTFILES:77: unrecognized option
tar: /tmp/ROOTFILES:78: unrecognized option
tar: /tmp/ROOTFILES:79: unrecognized option
tar: /tmp/ROOTFILES:80: unrecognized option
tar: /tmp/ROOTFILES:81: unrecognized option
tar: /tmp/ROOTFILES:82: unrecognized option
tar: /tmp/ROOTFILES:83: unrecognized option
tar: /tmp/ROOTFILES:84: unrecognized option
tar: /tmp/ROOTFILES:85: unrecognized option
tar: /tmp/ROOTFILES:86: unrecognized option
tar: /tmp/ROOTFILES:87: unrecognized option
tar: Exiting with failure status due to previous errors
make: *** [cdrom:158: /usr/src/log/cdrom] Error 2

Re: Errors at cdrom stage when building the python3 update package

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]

Hello Adolf,

Interesting problem.

It might be that you have some lines in any of the root files starting with “-“.

Therefore tar might be interpreting this as command line options, but not recognise them.

Does this help?

-Michael

> On 15 Oct 2022, at 20:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
> 
> Hi everyone,
> 
> I have been working on the python3 update. With python3 updated then some of the individual python packages required to be updated as they failed to build.
> 
> 
> To make it simpler I updated all the python3 packages that had new versions and this also resulted in some rust packages needing to be updated. After several rounds then everything built without any problems until it came to the cdrom stage.
> 
> 
> I got the following errors and I have no idea what I have done wrong to make this happen. The error messages unfortunately aren't much help to me.
> 
> Can anyone help with what the issue is causing the attached log file errors.
> 
> 
> Regards,
> 
> Adolf.
> <_build.packages.log>

Re: Errors at cdrom stage when building the python3 update package

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

Hi Michael,

On 17/10/2022 16:08, Michael Tremer wrote:
> Hello Adolf,
> 
> Interesting problem.
> 
> It might be that you have some lines in any of the root files starting with “-“.
> 
> Therefore tar might be interpreting this as command line options, but not recognise them.
> 
> Does this help?
Absolutely.

I checked most of the rootfiles for both +'s and -'s but going back and looking at them all again I found three with -'s that I had not removed. One of those was the python3 rootfile itself so it had 85 lines with - at the start.

I will re-do the build again after fixing those three rootfiles and I expect it to work then.

Thanks very much for the help.

Regards,
Adolf.
> 
> -Michael
> 
>> On 15 Oct 2022, at 20:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>
>> Hi everyone,
>>
>> I have been working on the python3 update. With python3 updated then some of the individual python packages required to be updated as they failed to build.
>>
>>
>> To make it simpler I updated all the python3 packages that had new versions and this also resulted in some rust packages needing to be updated. After several rounds then everything built without any problems until it came to the cdrom stage.
>>
>>
>> I got the following errors and I have no idea what I have done wrong to make this happen. The error messages unfortunately aren't much help to me.
>>
>> Can anyone help with what the issue is causing the attached log file errors.
>>
>>
>> Regards,
>>
>> Adolf.
>> <_build.packages.log>
> 

Re: Errors at cdrom stage when building the python3 update package

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 4579 bytes --]

Hi Michael and all,

On 17/10/2022 17:42, Adolf Belka wrote:
> Hi Michael,
>
> On 17/10/2022 16:08, Michael Tremer wrote:
>> Hello Adolf,
>>
>> Interesting problem.
>>
>> It might be that you have some lines in any of the root files starting with “-“.
>>
>> Therefore tar might be interpreting this as command line options, but not recognise them.
>>
>> Does this help?
> Absolutely.
>
> I checked most of the rootfiles for both +'s and -'s but going back and looking at them all again I found three with -'s that I had not removed. One of those was the python3 rootfile itself so it had 85 lines with - at the start.
>
> I will re-do the build again after fixing those three rootfiles and I expect it to work then.
>
That problem was solved but I have now found three programs that have not been updated where the change of python from 10.3.1 to 10.3.8 has changed their rootfiles significantly.

libplist has lost one line in the log rootfile

#usr/bin/plistutil
#usr/include/plist
#usr/include/plist/Array.h
#usr/include/plist/Boolean.h
#usr/include/plist/Data.h
#usr/include/plist/Date.h
#usr/include/plist/Dictionary.h
#usr/include/plist/Integer.h
#usr/include/plist/Key.h
#usr/include/plist/Node.h
#usr/include/plist/Real.h
#usr/include/plist/String.h
#usr/include/plist/Structure.h
#usr/include/plist/Uid.h
#usr/include/plist/plist++.h
#usr/include/plist/plist.h
#usr/lib/libplist++-2.0.la
#usr/lib/libplist++-2.0.so
usr/lib/libplist++-2.0.so.3
usr/lib/libplist++-2.0.so.3.3.0
#usr/lib/libplist-2.0.la
#usr/lib/libplist-2.0.so
usr/lib/libplist-2.0.so.3
usr/lib/libplist-2.0.so.3.3.0
#usr/lib/pkgconfig/libplist++-2.0.pc
#usr/lib/pkgconfig/libplist-2.0.pc
#usr/share/man/man1/plistutil.1
-usr/lib/python3.10/site-packages/plist.so

I can't tell if this is a problem or not. It was the only file in site-packages for this program.


iotop has the following list in the log rootfile

+usr/bin/iotop
+usr/lib/python3.10/site-packages/iotop-0.6-py3.10.egg
-usr/lib/python3.10/site-packages/iotop
-usr/sbin/iotop


All the 10 previously commented out lines are gone.

the iotop binary has moved from sbin to bin. This is despite the section in setup.py labelled as a dirty hack to ensure iotop is put in sbin instead of bin. That no longer seems to work with python3.10.8

Not sure how to fix this in setup.py. There is nothing about this in the iotop git repository.

I could just adjust the lfs file to move iotop from bin to sbin after the install has been run but not sure if that is the best thing to do or not.



python3-flit has added the following lines in the log rootfile

+root/.cache
+root/.cache/pip
+root/.cache/pip/selfcheck
+root/.cache/pip/selfcheck/241e69162522ccf5846a2f42ebc24b17464915a155679666b89a9f31


I have never seen anything like this before and it definitely looks wrong but I don't know what to do to fix it. There is no new update of flit that might fix it

flit is built using pip so maybe something changed between python-3.10.1 and python-3.10.8 with pip but I couldn't find anything obvious looking through the change logs and I don't understand all the commands in the lfs file for flit to be able to try and adjust it.


Any suggestion would be gladly accepted for how to deal with the above three packages.


Once these three are dealt with then that should be it for the python3 update as the build got to the end without any further flags about rootfiles or anything else.  Then I would need to do some testing to make sure IPFire still works before I submit a large patch set again.


Regards,


Adolf.



> Thanks very much for the help.
>
> Regards,
> Adolf.
>>
>> -Michael
>>
>>> On 15 Oct 2022, at 20:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>
>>> Hi everyone,
>>>
>>> I have been working on the python3 update. With python3 updated then some of the individual python packages required to be updated as they failed to build.
>>>
>>>
>>> To make it simpler I updated all the python3 packages that had new versions and this also resulted in some rust packages needing to be updated. After several rounds then everything built without any problems until it came to the cdrom stage.
>>>
>>>
>>> I got the following errors and I have no idea what I have done wrong to make this happen. The error messages unfortunately aren't much help to me.
>>>
>>> Can anyone help with what the issue is causing the attached log file errors.
>>>
>>>
>>> Regards,
>>>
>>> Adolf.
>>> <_build.packages.log>
>>

Re: Errors at cdrom stage when building the python3 update package

[Adolf Belka]

[-- Attachment #1: Type: text/plain, Size: 5644 bytes --]

Hi All,

Resolved all remaining build errors and installed the iso that was created and tested it on a vm. Everything that I tested on the IPFire vm worked fine.

Only thing I couldn't completely test was the modification of libplist with shairport-sync as I do not have any Apple/iOS products at all. However shairport-sync started and stopped without any error messages so I am hopeful that the required dependency of libplist works with it.

Also good news, I used the all option for the git send-email command and everything was sent successfully without any problems with the number of sends. That made it much easier and quicker.

I have also uploaded the python-3.10.8 branch to my IPFire repository.

https://git.ipfire.org/?p=people/bonnietwin/ipfire-2.x.git;a=shortlog;h=refs/heads/python-3.10.8

Regards,

Adolf.


On 18/10/2022 14:31, Adolf Belka wrote:
> Hi Michael and all,
>
> On 17/10/2022 17:42, Adolf Belka wrote:
>> Hi Michael,
>>
>> On 17/10/2022 16:08, Michael Tremer wrote:
>>> Hello Adolf,
>>>
>>> Interesting problem.
>>>
>>> It might be that you have some lines in any of the root files starting with “-“.
>>>
>>> Therefore tar might be interpreting this as command line options, but not recognise them.
>>>
>>> Does this help?
>> Absolutely.
>>
>> I checked most of the rootfiles for both +'s and -'s but going back and looking at them all again I found three with -'s that I had not removed. One of those was the python3 rootfile itself so it had 85 lines with - at the start.
>>
>> I will re-do the build again after fixing those three rootfiles and I expect it to work then.
>>
> That problem was solved but I have now found three programs that have not been updated where the change of python from 10.3.1 to 10.3.8 has changed their rootfiles significantly.
>
> libplist has lost one line in the log rootfile
>
> #usr/bin/plistutil
> #usr/include/plist
> #usr/include/plist/Array.h
> #usr/include/plist/Boolean.h
> #usr/include/plist/Data.h
> #usr/include/plist/Date.h
> #usr/include/plist/Dictionary.h
> #usr/include/plist/Integer.h
> #usr/include/plist/Key.h
> #usr/include/plist/Node.h
> #usr/include/plist/Real.h
> #usr/include/plist/String.h
> #usr/include/plist/Structure.h
> #usr/include/plist/Uid.h
> #usr/include/plist/plist++.h
> #usr/include/plist/plist.h
> #usr/lib/libplist++-2.0.la
> #usr/lib/libplist++-2.0.so
> usr/lib/libplist++-2.0.so.3
> usr/lib/libplist++-2.0.so.3.3.0
> #usr/lib/libplist-2.0.la
> #usr/lib/libplist-2.0.so
> usr/lib/libplist-2.0.so.3
> usr/lib/libplist-2.0.so.3.3.0
> #usr/lib/pkgconfig/libplist++-2.0.pc
> #usr/lib/pkgconfig/libplist-2.0.pc
> #usr/share/man/man1/plistutil.1
> -usr/lib/python3.10/site-packages/plist.so
>
> I can't tell if this is a problem or not. It was the only file in site-packages for this program.
>
>
> iotop has the following list in the log rootfile
>
> +usr/bin/iotop
> +usr/lib/python3.10/site-packages/iotop-0.6-py3.10.egg
> -usr/lib/python3.10/site-packages/iotop
> -usr/sbin/iotop
>
>
> All the 10 previously commented out lines are gone.
>
> the iotop binary has moved from sbin to bin. This is despite the section in setup.py labelled as a dirty hack to ensure iotop is put in sbin instead of bin. That no longer seems to work with python3.10.8
>
> Not sure how to fix this in setup.py. There is nothing about this in the iotop git repository.
>
> I could just adjust the lfs file to move iotop from bin to sbin after the install has been run but not sure if that is the best thing to do or not.
>
>
>
> python3-flit has added the following lines in the log rootfile
>
> +root/.cache
> +root/.cache/pip
> +root/.cache/pip/selfcheck
> +root/.cache/pip/selfcheck/241e69162522ccf5846a2f42ebc24b17464915a155679666b89a9f31
>
>
> I have never seen anything like this before and it definitely looks wrong but I don't know what to do to fix it. There is no new update of flit that might fix it
>
> flit is built using pip so maybe something changed between python-3.10.1 and python-3.10.8 with pip but I couldn't find anything obvious looking through the change logs and I don't understand all the commands in the lfs file for flit to be able to try and adjust it.
>
>
> Any suggestion would be gladly accepted for how to deal with the above three packages.
>
>
> Once these three are dealt with then that should be it for the python3 update as the build got to the end without any further flags about rootfiles or anything else.  Then I would need to do some testing to make sure IPFire still works before I submit a large patch set again.
>
>
> Regards,
>
>
> Adolf.
>
>
>
>> Thanks very much for the help.
>>
>> Regards,
>> Adolf.
>>>
>>> -Michael
>>>
>>>> On 15 Oct 2022, at 20:25, Adolf Belka <adolf.belka(a)ipfire.org> wrote:
>>>>
>>>> Hi everyone,
>>>>
>>>> I have been working on the python3 update. With python3 updated then some of the individual python packages required to be updated as they failed to build.
>>>>
>>>>
>>>> To make it simpler I updated all the python3 packages that had new versions and this also resulted in some rust packages needing to be updated. After several rounds then everything built without any problems until it came to the cdrom stage.
>>>>
>>>>
>>>> I got the following errors and I have no idea what I have done wrong to make this happen. The error messages unfortunately aren't much help to me.
>>>>
>>>> Can anyone help with what the issue is causing the attached log file errors.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Adolf.
>>>> <_build.packages.log>
>>>