Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Hello *,
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this issue. When the pakfire-builder binary is started it directly get killed by a SEGV.
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
At this stage the pakfire-builder works fine again and is able to build packages. But I found an additional issue when using this git state:
https://bugzilla.ipfire.org/show_bug.cgi?id=13362
Can you please check, if you are affected by this issue too?
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Best regards,
-Stefan
Hi Stefan,
On 30/09/2023 20:05, Stefan Schantl wrote:
Hello *,
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this issue. When the pakfire-builder binary is started it directly get killed by a SEGV.
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
At this stage the pakfire-builder works fine again and is able to build packages. But I found an additional issue when using this git state:
https://bugzilla.ipfire.org/show_bug.cgi?id=13362
Can you please check, if you are affected by this issue too?
I have added my input into the bug report.
Regards, Adolf
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Best regards,
-Stefan
Hi Michael,
On 01/10/2023 12:24, Adolf Belka wrote:
Hi Stefan,
On 30/09/2023 20:05, Stefan Schantl wrote:
Hello *,
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this issue. When the pakfire-builder binary is started it directly get killed by a SEGV.
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
So far running a build of any existing packages seems to work without a problem. I am then creating a python3-build-0.9.0 build as that version still has setup.py to use that as the bootstrap for the newer python3-build package.
Tested running python3-build-0.9.0 and it goes to download the package and then has a segmentation fault.
I changed the package name to just python3-build but the same segmentation fault occurred.
Maybe I need to roll back to an even earlier commit of the git branch.
Regards, Adolf.
At this stage the pakfire-builder works fine again and is able to build packages. But I found an additional issue when using this git state:
https://bugzilla.ipfire.org/show_bug.cgi?id=13362
Can you please check, if you are affected by this issue too?
I have added my input into the bug report.
Regards, Adolf
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Best regards,
-Stefan
Hi All,
On 01/10/2023 13:36, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 12:24, Adolf Belka wrote:
Hi Stefan,
On 30/09/2023 20:05, Stefan Schantl wrote:
Hello *,
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this issue. When the pakfire-builder binary is started it directly get killed by a SEGV.
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
So far running a build of any existing packages seems to work without a problem. I am then creating a python3-build-0.9.0 build as that version still has setup.py to use that as the bootstrap for the newer python3-build package.
Tested running python3-build-0.9.0 and it goes to download the package and then has a segmentation fault.
I changed the package name to just python3-build but the same segmentation fault occurred.
Maybe I need to roll back to an even earlier commit of the git branch.
I think I do need to roll back further. After getting a segmentation fault every time I tried to build the python3-build package I had created, I made another new package, python3-botocore, just to test out.
When I ran the pakfire-builder it started downloading the package but then at 75% download everything on the laptop froze and I had to hard power off the laptop to get it to reset.
I will try a further rollback and try the new packages again and report back what I find.
Regards, Adolf.
Regards, Adolf.
At this stage the pakfire-builder works fine again and is able to build packages. But I found an additional issue when using this git state:
https://bugzilla.ipfire.org/show_bug.cgi?id=13362
Can you please check, if you are affected by this issue too?
I have added my input into the bug report.
Regards, Adolf
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Best regards,
-Stefan
On 30 Sep 2023, at 19:05, Stefan Schantl stefan.schantl@ipfire.org wrote:
Hello *,
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this issue. When the pakfire-builder binary is started it directly get killed by a SEGV.
Is this on Debian? It works fine for me on Debian Bookworm.
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
At this stage the pakfire-builder works fine again and is able to build packages. But I found an additional issue when using this git state:
https://bugzilla.ipfire.org/show_bug.cgi?id=13362
Can you please check, if you are affected by this issue too?
I cannot reproduce this problem whatsoever.
Let me know if there is anything else I should try to narrow the problem down.
Regards, Adolf.
Best regards,
-Stefan
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
-- Sent from my laptop
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I just ran git pull which took me to commit
build: Add implicit dist() when a makefile is passed -- followed by make and make install.
Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
Rolling back to
cli: pakfire-builder: Implement "image create"
then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session.
Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop.
So rolling back to
cli: pakfire-builder: Implement "image create"
Does not stop the problem happening it just seems to show itself in different forms.
I have now rolled back to
cli: pakfire: Implement --yes
which I think is where my last git pull had placed me.
Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good.
Let me know if there is anything else I should try to narrow the problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
Regards, Adolf.
-- Sent from my laptop
-- Sent from my laptop
Trying resend of this message. I have not had a copy of this mail sent to me after nearly 4 hours, which I should have done as the IPFire dev list is on the cc.
Maybe something is still not working well on the mail server.
Regards, Adolf.
On 06/10/2023 19:11, Adolf Belka wrote:
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
> On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org > wrote: > > Hi Michael, > > I just ran git pull which took me to commit > > build: Add implicit dist() when a makefile is passed -- followed > by make and make install. > > Running pakfire-builder build beep/beep.nm gave the error > message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
> Rolling back to > > cli: pakfire-builder: Implement "image create" > > then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
> Then tried git pull, make, make install again and this time beep > built without any problems. Then I tried sqlite and I got the > segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
> Then I rolled back and sqlite built okay. Then ran sqlite again > and this time part way through building my laptop logged out > from my session. > > Logged back in and ran sqlite again and this after a short while > everything froze and I had to reset the laptop. > > So rolling back to > > cli: pakfire-builder: Implement "image create" > > Does not stop the problem happening it just seems to show itself > in different forms. > > I have now rolled back to > > cli: pakfire: Implement --yes > > which I think is where my last git pull had placed me. > > Running pakfire-builder on beep twice and sqlite four times has > given no problems so that commit stage is confirmed good. > > Let me know if there is anything else I should try to narrow the > problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
> > Regards, > Adolf. > > > -- > Sent from my laptop >
-- Sent from my laptop
Hi Michael,
I tried to see if gdb would show anything about what was happening.
I ran gdb pakfire-builder and then within gdb ran the command run build beep/beep.nm
Here is the output from the gdb pakfire-builder command running as root.I can't really figure out what is happening, except that pakfire-builder seems to have exited very early
gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build beep/beep.nm Starting program: /usr/bin/pakfire-builder build beep/beep.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff57e16c0 (LWP 1342)] [Thread 0x7ffff57e16c0 (LWP 1342) exited] [Inferior 1 (process 1338) exited with code 0377] (gdb)
Regards, Adolf
On 06/10/2023 19:11, Adolf Belka wrote:
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote:
Hello,
> On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org > wrote: > > Hi Michael, > > I just ran git pull which took me to commit > > build: Add implicit dist() when a makefile is passed -- followed > by make and make install. > > Running pakfire-builder build beep/beep.nm gave the error > message "Segmentation fault"
I can confirm this, but it seems to be a crash in libcurl/glibc:
> Rolling back to > > cli: pakfire-builder: Implement "image create" > > then worked with beep fine.
However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same.
My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf.
In IPFire we use c-ares as a resolver for cURL which does not crash like this.
> Then tried git pull, make, make install again and this time beep > built without any problems. Then I tried sqlite and I got the > segmentation fault again.
This is very early when Pakfire is starting up and refreshing its repository information.
> Then I rolled back and sqlite built okay. Then ran sqlite again > and this time part way through building my laptop logged out > from my session. > > Logged back in and ran sqlite again and this after a short while > everything froze and I had to reset the laptop. > > So rolling back to > > cli: pakfire-builder: Implement "image create" > > Does not stop the problem happening it just seems to show itself > in different forms. > > I have now rolled back to > > cli: pakfire: Implement --yes > > which I think is where my last git pull had placed me. > > Running pakfire-builder on beep twice and sqlite four times has > given no problems so that commit stage is confirmed good. > > Let me know if there is anything else I should try to narrow the > problem down.
Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this…
-Michael
> > Regards, > Adolf. > > > -- > Sent from my laptop >
-- Sent from my laptop
Hello Adolf,
Thank you for giving this another go :)
I am currently sidetracked with a couple of other things (will send an email about this hopefully soon), and there might be some more problems in Pakfire that I might have recently introduced.
However, it exists normally, but with an error code. Normally it should log something to syslog. If not, please run the same thing with —debug and if it still does not tell you why it is unhappy, let me know.
Best, -Michael
On 11 Oct 2023, at 14:08, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I tried to see if gdb would show anything about what was happening.
I ran gdb pakfire-builder and then within gdb ran the command run build beep/beep.nm
Here is the output from the gdb pakfire-builder command running as root.I can't really figure out what is happening, except that pakfire-builder seems to have exited very early
gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build beep/beep.nm Starting program: /usr/bin/pakfire-builder build beep/beep.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff57e16c0 (LWP 1342)] [Thread 0x7ffff57e16c0 (LWP 1342) exited] [Inferior 1 (process 1338) exited with code 0377] (gdb)
Regards, Adolf
On 06/10/2023 19:11, Adolf Belka wrote:
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote:
Hi Michael,
On 01/10/2023 14:22, Michael Tremer wrote: > Hello, > >> On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote: >> >> Hi Michael, >> >> I just ran git pull which took me to commit >> >> build: Add implicit dist() when a makefile is passed -- followed by make and make install. >> >> Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault" > > I can confirm this, but it seems to be a crash in libcurl/glibc: > > > >> Rolling back to >> >> cli: pakfire-builder: Implement "image create" >> >> then worked with beep fine. > > However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same. My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again.
Then I rolled back a long way and tried again with the python3-build and again it segfaulted.
So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault.
Regards, Adolf. > > In IPFire we use c-ares as a resolver for cURL which does not crash like this. > >> Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again. > > This is very early when Pakfire is starting up and refreshing its repository information. > >> Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session. >> >> Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop. >> >> So rolling back to >> >> cli: pakfire-builder: Implement "image create" >> >> Does not stop the problem happening it just seems to show itself in different forms. >> >> I have now rolled back to >> >> cli: pakfire: Implement --yes >> >> which I think is where my last git pull had placed me. >> >> Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good. >> >> Let me know if there is anything else I should try to narrow the problem down. > > Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this… > > -Michael > >> >> Regards, >> Adolf. >> >> >> -- >> Sent from my laptop >> >
-- Sent from my laptop
-- Sent from my laptop
Hi Michael,
Duh, didn't think of looking in the syslog.
Ran the pakfire-builder command and got the following output in the arch linux journalctl
Oct 11 16:22:33 ipfire pakfire[1304]: Could not link destination file /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json: Invalid cross-device link Oct 11 16:22:33 ipfire systemd[1]: var-tmp-pakfire\x2droot\x2dnkqHr7.mount: Deactivated successfully.
Then ran the command including --debug and got the following:-
Oct 11 16:18:32 ipfire pakfire[1018]: Mounting /var/tmp/pakfire-root-ewgZuw from pakfire_ramdisk (tmpfs - (null)) Oct 11 16:18:32 ipfire pakfire[1018]: Ramdisk mounted at /var/tmp/pakfire-root-ewgZuw ((null)) Oct 11 16:18:32 ipfire pakfire[1018]: Fetching SUBID from /etc/subuid for root Oct 11 16:18:32 ipfire pakfire[1018]: Fetching SUBID from /etc/subgid for root Oct 11 16:18:32 ipfire pakfire[1018]: RLIMIT_NOFILE set to 524288 Oct 11 16:18:32 ipfire pakfire[1018]: Pakfire initialized at 0x55555555f950 Oct 11 16:18:32 ipfire pakfire[1018]: user = root (0) Oct 11 16:18:32 ipfire pakfire[1018]: group = root (0) Oct 11 16:18:32 ipfire pakfire[1018]: arch = x86_64 (x86_64) Oct 11 16:18:32 ipfire pakfire[1018]: path = /var/tmp/pakfire-root-ewgZuw Oct 11 16:18:32 ipfire pakfire[1018]: Reading configuration Oct 11 16:18:32 ipfire pakfire[1018]: Reading repository configuration from /var/tmp/pakfire-root-ewgZuw/etc/pakfire/repos Oct 11 16:18:32 ipfire pakfire[1018]: Distribution: IPFire 3 Oct 11 16:18:32 ipfire pakfire[1018]: name = IPFire Oct 11 16:18:32 ipfire pakfire[1018]: id = ipfire Oct 11 16:18:32 ipfire pakfire[1018]: version = 3 Oct 11 16:18:32 ipfire pakfire[1018]: version_id = 3 Oct 11 16:18:32 ipfire pakfire[1018]: vendor = IPFire Project Oct 11 16:18:32 ipfire pakfire[1018]: slogan = An Open Source Firewall Solution Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA foreign_keys = ON Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA case_sensitive_like = ON Oct 11 16:18:32 ipfire pakfire[1018]: Could not find last modification timestamp Oct 11 16:18:32 ipfire pakfire[1018]: The database was last modified at 0 Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA secure_delete = OFF Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA journal_mode = WAL Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: BEGIN TRANSACTION Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS settings(key TEXT, val TEXT) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE UNIQUE INDEX IF NOT EXISTS settings_key ON settings(key) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS packages(id INTEGER PRIMARY KEY, name TEXT, evr TEXT, arch TEXT, groups TEXT, filename TEXT, size INTEGER, inst_size INTEGER, digest_type INTEGER, digest BLOB, license TEXT, summary TEXT, description TEXT, uuid TEXT, vendor TEXT, build_host TEXT, build_time INTEGER, installed INTEGER, userinstalled INTEGER, repository TEXT, source_name TEXT, source_evr TEXT, source_arch TEXT, distribution TEXT) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS packages_name ON packages(name) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE UNIQUE INDEX IF NOT EXISTS packages_uuid ON packages(uuid) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS dependencies(pkg INTEGER, type TEXT, dependency TEXT, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS dependencies_pkg_index ON dependencies(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS files(id INTEGER PRIMARY KEY, path TEXT, pkg INTEGER, size INTEGER, config INTEGER, datafile INTEGER, mode INTEGER, uname TEXT, gname TEXT, ctime INTEGER, mtime INTEGER, mimetype TEXT, capabilities TEXT, digest_sha2_512 BLOB, digest_sha2_256 BLOB, digest_blake2b512 BLOB, digest_blake2s256 BLOB, digest_sha3_512 BLOB, digest_sha3_256 BLOB, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS files_pkg_index ON files(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS files_path_index ON files(path) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS scriptlets(id INTEGER PRIMARY KEY, pkg INTEGER, type TEXT, scriptlet TEXT, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS scriptlets_pkg_index ON scriptlets(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Setting arch to 'x86_64' Oct 11 16:18:32 ipfire pakfire[1018]: Setting schema to '8' Oct 11 16:18:32 ipfire pakfire[1018]: Setting last_modified_at to '1697033912' Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: COMMIT Oct 11 16:18:32 ipfire pakfire[1018]: Loading package database... Oct 11 16:18:32 ipfire pakfire[1018]: Loading package database completed in 0.3750ms Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA optimize Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA wal_checkpoint = TRUNCATE Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository stable Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository testing Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository local Oct 11 16:18:32 ipfire pakfire[1018]: Repository URL updated: Oct 11 16:18:32 ipfire pakfire[1018]: From: https://pakfire.ipfire.org/files/repos/ipfire-3/stable/%%7Barch%7D Oct 11 16:18:32 ipfire pakfire[1018]: To : https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Trying 81.3.27.49:443... Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Connected to pakfire.ipfire.org (81.3.27.49) port 443 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: ALPN: curl offers h2,http/1.1 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS handshake, Client hello (1): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: CAfile: /etc/ssl/certs/ca-certificates.crt Oct 11 16:18:32 ipfire pakfire[1018]: cURL: CApath: none Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Server hello (2): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Certificate (11): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, CERT verify (15): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Finished (20): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS handshake, Finished (20): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: ALPN: server accepted h2 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Server certificate: Oct 11 16:18:32 ipfire pakfire[1018]: cURL: subject: CN=pakfire.ipfire.org Oct 11 16:18:32 ipfire pakfire[1018]: cURL: start date: Sep 8 22:29:55 2023 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: expire date: Dec 7 22:29:54 2023 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: subjectAltName: host "pakfire.ipfire.org" matched cert's "pakfire.ipfire.org" Oct 11 16:18:32 ipfire pakfire[1018]: cURL: issuer: C=US; O=Let's Encrypt; CN=R3 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: SSL certificate verify ok. Oct 11 16:18:32 ipfire pakfire[1018]: cURL: using HTTP/2 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] OPENED stream for https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64/repodata/repom... Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:method: GET] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:scheme: https] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:authority: pakfire.ipfire.org] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:path: /files/repos/ipfire-3/stable/x86_64/repodata/repomd.json] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [user-agent: pakfire/0.9.29] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [accept: */*] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: > GET /files/repos/ipfire-3/stable/x86_64/repodata/repomd.json HTTP/2 Host: pakfire.ipfire.org User-Agent: pakfire/0.9.29 Accept: */* Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: old SSL session ID is stale, removing Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < HTTP/2 200 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < date: Wed, 11 Oct 2023 14:18:32 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < last-modified: Tue, 19 Sep 2023 17:14:55 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < etag: "5e-605b965d1d454" Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < accept-ranges: bytes Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < content-length: 94 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-content-type-options: nosniff Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-frame-options: deny Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < referrer-policy: strict-origin Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-xss-protection: 1; mode=block Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < content-type: application/json Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < strict-transport-security: max-age=31536000; includeSubDomains; preload Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Connection #0 to host pakfire.ipfire.org left intact Oct 11 16:18:32 ipfire pakfire[1018]: cURL transfer done: 0 - No error Oct 11 16:18:32 ipfire pakfire[1018]: Effective URL: https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64/repodata/repom... Oct 11 16:18:32 ipfire pakfire[1018]: Response code: 200 Oct 11 16:18:32 ipfire pakfire[1018]: HTTP Version: HTTP/2.0 Oct 11 16:18:32 ipfire pakfire[1018]: Total Time: 0.11s Oct 11 16:18:32 ipfire pakfire[1018]: Download Size: 94 bytes Oct 11 16:18:32 ipfire pakfire[1018]: Download Speed: 867 bps Oct 11 16:18:32 ipfire pakfire[1018]: Message Digest: Oct 11 16:18:32 ipfire pakfire[1018]: Download successful. Storing result in /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json Oct 11 16:18:32 ipfire pakfire[1018]: Could not link destination file /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json: Invalid cross-device link Oct 11 16:18:32 ipfire pakfire[1018]: Destroying /var/tmp/pakfire-root-ewgZuw Oct 11 16:18:32 ipfire systemd[1]: var-tmp-pakfire\x2droot\x2dewgZuw.mount: Deactivated successfully.
Regards, Adolf.
On 11/10/2023 15:34, Michael Tremer wrote:
Hello Adolf,
Thank you for giving this another go :)
I am currently sidetracked with a couple of other things (will send an email about this hopefully soon), and there might be some more problems in Pakfire that I might have recently introduced.
However, it exists normally, but with an error code. Normally it should log something to syslog. If not, please run the same thing with —debug and if it still does not tell you why it is unhappy, let me know.
Best, -Michael
On 11 Oct 2023, at 14:08, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I tried to see if gdb would show anything about what was happening.
I ran gdb pakfire-builder and then within gdb ran the command run build beep/beep.nm
Here is the output from the gdb pakfire-builder command running as root.I can't really figure out what is happening, except that pakfire-builder seems to have exited very early
gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build beep/beep.nm Starting program: /usr/bin/pakfire-builder build beep/beep.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff57e16c0 (LWP 1342)] [Thread 0x7ffff57e16c0 (LWP 1342) exited] [Inferior 1 (process 1338) exited with code 0377] (gdb)
Regards, Adolf
On 06/10/2023 19:11, Adolf Belka wrote:
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote:
Hi Michael,
I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package.
The output I got was
sudo gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build python3-build/python3-build.nm Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff55276c0 (LWP 61317)]
Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff55276c0 (LWP 61317)] 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 (gdb)
I see libcurl and libc present there so that looks to be what you indicated.
I have curl-8.3.0 and glibc-2.38 on my system.
Regards,
Adolf.
On 01/10/2023 14:51, Adolf Belka wrote: > Hi Michael, > > On 01/10/2023 14:22, Michael Tremer wrote: >> Hello, >> >>> On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote: >>> >>> Hi Michael, >>> >>> I just ran git pull which took me to commit >>> >>> build: Add implicit dist() when a makefile is passed -- followed by make and make install. >>> >>> Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault" >> >> I can confirm this, but it seems to be a crash in libcurl/glibc: >> >> >> >>> Rolling back to >>> >>> cli: pakfire-builder: Implement "image create" >>> >>> then worked with beep fine. >> >> However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same. > My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again. > > Then I rolled back a long way and tried again with the python3-build and again it segfaulted. > > So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault. > > Regards, > Adolf. >> >> In IPFire we use c-ares as a resolver for cURL which does not crash like this. >> >>> Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again. >> >> This is very early when Pakfire is starting up and refreshing its repository information. >> >>> Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session. >>> >>> Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop. >>> >>> So rolling back to >>> >>> cli: pakfire-builder: Implement "image create" >>> >>> Does not stop the problem happening it just seems to show itself in different forms. >>> >>> I have now rolled back to >>> >>> cli: pakfire: Implement --yes >>> >>> which I think is where my last git pull had placed me. >>> >>> Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good. >>> >>> Let me know if there is anything else I should try to narrow the problem down. >> >> Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this… >> >> -Michael >> >>> >>> Regards, >>> Adolf. >>> >>> >>> -- >>> Sent from my laptop >>> >> >
-- Sent from my laptop
-- Sent from my laptop
Hello Adolf,
On 11 Oct 2023, at 15:27, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
Duh, didn't think of looking in the syslog.
Ran the pakfire-builder command and got the following output in the arch linux journalctl
Oct 11 16:22:33 ipfire pakfire[1304]: Could not link destination file /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json: Invalid cross-device link Oct 11 16:22:33 ipfire systemd[1]: var-tmp-pakfire\x2droot\x2dnkqHr7.mount: Deactivated successfully.
This is also now finally solved:
https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=90a6b8e4f704c3c628e810b...
Then ran the command including --debug and got the following:-
Oct 11 16:18:32 ipfire pakfire[1018]: Mounting /var/tmp/pakfire-root-ewgZuw from pakfire_ramdisk (tmpfs - (null)) Oct 11 16:18:32 ipfire pakfire[1018]: Ramdisk mounted at /var/tmp/pakfire-root-ewgZuw ((null)) Oct 11 16:18:32 ipfire pakfire[1018]: Fetching SUBID from /etc/subuid for root Oct 11 16:18:32 ipfire pakfire[1018]: Fetching SUBID from /etc/subgid for root Oct 11 16:18:32 ipfire pakfire[1018]: RLIMIT_NOFILE set to 524288 Oct 11 16:18:32 ipfire pakfire[1018]: Pakfire initialized at 0x55555555f950 Oct 11 16:18:32 ipfire pakfire[1018]: user = root (0) Oct 11 16:18:32 ipfire pakfire[1018]: group = root (0) Oct 11 16:18:32 ipfire pakfire[1018]: arch = x86_64 (x86_64) Oct 11 16:18:32 ipfire pakfire[1018]: path = /var/tmp/pakfire-root-ewgZuw Oct 11 16:18:32 ipfire pakfire[1018]: Reading configuration Oct 11 16:18:32 ipfire pakfire[1018]: Reading repository configuration from /var/tmp/pakfire-root-ewgZuw/etc/pakfire/repos Oct 11 16:18:32 ipfire pakfire[1018]: Distribution: IPFire 3 Oct 11 16:18:32 ipfire pakfire[1018]: name = IPFire Oct 11 16:18:32 ipfire pakfire[1018]: id = ipfire Oct 11 16:18:32 ipfire pakfire[1018]: version = 3 Oct 11 16:18:32 ipfire pakfire[1018]: version_id = 3 Oct 11 16:18:32 ipfire pakfire[1018]: vendor = IPFire Project Oct 11 16:18:32 ipfire pakfire[1018]: slogan = An Open Source Firewall Solution Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA foreign_keys = ON Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA case_sensitive_like = ON Oct 11 16:18:32 ipfire pakfire[1018]: Could not find last modification timestamp Oct 11 16:18:32 ipfire pakfire[1018]: The database was last modified at 0 Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA secure_delete = OFF Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA journal_mode = WAL Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: BEGIN TRANSACTION Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS settings(key TEXT, val TEXT) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE UNIQUE INDEX IF NOT EXISTS settings_key ON settings(key) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS packages(id INTEGER PRIMARY KEY, name TEXT, evr TEXT, arch TEXT, groups TEXT, filename TEXT, size INTEGER, inst_size INTEGER, digest_type INTEGER, digest BLOB, license TEXT, summary TEXT, description TEXT, uuid TEXT, vendor TEXT, build_host TEXT, build_time INTEGER, installed INTEGER, userinstalled INTEGER, repository TEXT, source_name TEXT, source_evr TEXT, source_arch TEXT, distribution TEXT) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS packages_name ON packages(name) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE UNIQUE INDEX IF NOT EXISTS packages_uuid ON packages(uuid) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS dependencies(pkg INTEGER, type TEXT, dependency TEXT, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS dependencies_pkg_index ON dependencies(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS files(id INTEGER PRIMARY KEY, path TEXT, pkg INTEGER, size INTEGER, config INTEGER, datafile INTEGER, mode INTEGER, uname TEXT, gname TEXT, ctime INTEGER, mtime INTEGER, mimetype TEXT, capabilities TEXT, digest_sha2_512 BLOB, digest_sha2_256 BLOB, digest_blake2b512 BLOB, digest_blake2s256 BLOB, digest_sha3_512 BLOB, digest_sha3_256 BLOB, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS files_pkg_index ON files(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS files_path_index ON files(path) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE TABLE IF NOT EXISTS scriptlets(id INTEGER PRIMARY KEY, pkg INTEGER, type TEXT, scriptlet TEXT, FOREIGN KEY (pkg) REFERENCES packages(id) ON DELETE CASCADE) Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: CREATE INDEX IF NOT EXISTS scriptlets_pkg_index ON scriptlets(pkg) Oct 11 16:18:32 ipfire pakfire[1018]: Setting arch to 'x86_64' Oct 11 16:18:32 ipfire pakfire[1018]: Setting schema to '8' Oct 11 16:18:32 ipfire pakfire[1018]: Setting last_modified_at to '1697033912' Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: COMMIT Oct 11 16:18:32 ipfire pakfire[1018]: Loading package database... Oct 11 16:18:32 ipfire pakfire[1018]: Loading package database completed in 0.3750ms Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA optimize Oct 11 16:18:32 ipfire pakfire[1018]: Executing database query: PRAGMA wal_checkpoint = TRUNCATE Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository stable Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository testing Oct 11 16:18:32 ipfire pakfire[1018]: Creating repository local Oct 11 16:18:32 ipfire pakfire[1018]: Repository URL updated: Oct 11 16:18:32 ipfire pakfire[1018]: From: https://pakfire.ipfire.org/files/repos/ipfire-3/stable/%%7Barch%7D Oct 11 16:18:32 ipfire pakfire[1018]: To : https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Trying 81.3.27.49:443... Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Connected to pakfire.ipfire.org (81.3.27.49) port 443 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: ALPN: curl offers h2,http/1.1 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS handshake, Client hello (1): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: CAfile: /etc/ssl/certs/ca-certificates.crt Oct 11 16:18:32 ipfire pakfire[1018]: cURL: CApath: none Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Server hello (2): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Certificate (11): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, CERT verify (15): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Finished (20): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (OUT), TLS handshake, Finished (20): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: ALPN: server accepted h2 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Server certificate: Oct 11 16:18:32 ipfire pakfire[1018]: cURL: subject: CN=pakfire.ipfire.org Oct 11 16:18:32 ipfire pakfire[1018]: cURL: start date: Sep 8 22:29:55 2023 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: expire date: Dec 7 22:29:54 2023 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: subjectAltName: host "pakfire.ipfire.org" matched cert's "pakfire.ipfire.org" Oct 11 16:18:32 ipfire pakfire[1018]: cURL: issuer: C=US; O=Let's Encrypt; CN=R3 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: SSL certificate verify ok. Oct 11 16:18:32 ipfire pakfire[1018]: cURL: using HTTP/2 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] OPENED stream for https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64/repodata/repom... Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:method: GET] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:scheme: https] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:authority: pakfire.ipfire.org] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [:path: /files/repos/ipfire-3/stable/x86_64/repodata/repomd.json] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [user-agent: pakfire/0.9.29] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: [HTTP/2] [1] [accept: */*] Oct 11 16:18:32 ipfire pakfire[1018]: cURL: > GET /files/repos/ipfire-3/stable/x86_64/repodata/repomd.json HTTP/2 Host: pakfire.ipfire.org User-Agent: pakfire/0.9.29 Accept: */* Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): Oct 11 16:18:32 ipfire pakfire[1018]: cURL: old SSL session ID is stale, removing Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < HTTP/2 200 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < date: Wed, 11 Oct 2023 14:18:32 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < last-modified: Tue, 19 Sep 2023 17:14:55 GMT Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < etag: "5e-605b965d1d454" Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < accept-ranges: bytes Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < content-length: 94 Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-content-type-options: nosniff Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-frame-options: deny Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < referrer-policy: strict-origin Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < x-xss-protection: 1; mode=block Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < content-type: application/json Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < strict-transport-security: max-age=31536000; includeSubDomains; preload Oct 11 16:18:32 ipfire pakfire[1018]: cURL: < Oct 11 16:18:32 ipfire pakfire[1018]: cURL: Connection #0 to host pakfire.ipfire.org left intact Oct 11 16:18:32 ipfire pakfire[1018]: cURL transfer done: 0 - No error Oct 11 16:18:32 ipfire pakfire[1018]: Effective URL: https://pakfire.ipfire.org/files/repos/ipfire-3/stable/x86_64/repodata/repom... Oct 11 16:18:32 ipfire pakfire[1018]: Response code: 200 Oct 11 16:18:32 ipfire pakfire[1018]: HTTP Version: HTTP/2.0 Oct 11 16:18:32 ipfire pakfire[1018]: Total Time: 0.11s Oct 11 16:18:32 ipfire pakfire[1018]: Download Size: 94 bytes Oct 11 16:18:32 ipfire pakfire[1018]: Download Speed: 867 bps Oct 11 16:18:32 ipfire pakfire[1018]: Message Digest: Oct 11 16:18:32 ipfire pakfire[1018]: Download successful. Storing result in /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json Oct 11 16:18:32 ipfire pakfire[1018]: Could not link destination file /root/.pakfire/cache/ipfire/3/x86_64/stable/repodata/repomd.json: Invalid cross-device link Oct 11 16:18:32 ipfire pakfire[1018]: Destroying /var/tmp/pakfire-root-ewgZuw Oct 11 16:18:32 ipfire systemd[1]: var-tmp-pakfire\x2droot\x2dewgZuw.mount: Deactivated successfully.
Regards, Adolf.
On 11/10/2023 15:34, Michael Tremer wrote:
Hello Adolf, Thank you for giving this another go :) I am currently sidetracked with a couple of other things (will send an email about this hopefully soon), and there might be some more problems in Pakfire that I might have recently introduced. However, it exists normally, but with an error code. Normally it should log something to syslog. If not, please run the same thing with —debug and if it still does not tell you why it is unhappy, let me know. Best, -Michael
On 11 Oct 2023, at 14:08, Adolf Belka adolf.belka@ipfire.org wrote:
Hi Michael,
I tried to see if gdb would show anything about what was happening.
I ran gdb pakfire-builder and then within gdb ran the command run build beep/beep.nm
Here is the output from the gdb pakfire-builder command running as root.I can't really figure out what is happening, except that pakfire-builder seems to have exited very early
gdb pakfire-builder GNU gdb (GDB) 13.2 Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: https://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.
For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from pakfire-builder... (gdb) run build beep/beep.nm Starting program: /usr/bin/pakfire-builder build beep/beep.nm [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". [New Thread 0x7ffff57e16c0 (LWP 1342)] [Thread 0x7ffff57e16c0 (LWP 1342) exited] [Inferior 1 (process 1338) exited with code 0377] (gdb)
Regards, Adolf
On 06/10/2023 19:11, Adolf Belka wrote:
Hi Michael,
On 06/10/2023 17:21, Michael Tremer wrote:
Hello everyone,
Finally there is some white smoke on Friday!
After a *very* long time, I found the reason why Pakfire was crashing as soon as a thread was launched (which is what actually happened). The reason is a compiler bug which remains unresolved since 2017: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81142
The fix for this is: https://git.ipfire.org/?p=pakfire.git;a=commitdiff;h=b4d087f3353a174be936da4...
Thank you for your patience. Please pull and let me know what new bugs I have created in the meantime.
I did a git pull and then ran the build commands but still ran into the progressbar vs progressbar2 issue.
I changed progressbar2 in configure.ac to progressbar and repeated the ./autogen etc and everything then went through without a problem.
Then I tried sudo pakfire-builder build beep/beep.nm and the program came straight back to a new line without appearing to do anything and without any messages at all.
I the did sudo pakfire-builder --debug build beep/beep.nm and exactly the same happened, went to a new line without any messages or anything.
Regards, Adolf.
Best, -Michael
On 1 Oct 2023, at 17:11, Adolf Belka adolf.belka@ipfire.org wrote:
I downgraded curl and glibc to the previous versions just to see if that stopped the segfault problem but it didn't
Regards,
Adolf.
On 01/10/2023 16:24, Adolf Belka wrote: > Hi Michael, > > I eventually figured out how to use gdb in this situation and managed to get a backtrace when running the build of the python3-build package. > > The output I got was > > sudo gdb pakfire-builder > GNU gdb (GDB) 13.2 > Copyright (C) 2023 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > Type "show copying" and "show warranty" for details. > This GDB was configured as "x86_64-pc-linux-gnu". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > https://www.gnu.org/software/gdb/bugs/. > Find the GDB manual and other documentation resources online at: > http://www.gnu.org/software/gdb/documentation/. > > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from pakfire-builder... > (gdb) run build python3-build/python3-build.nm > Starting program: /usr/bin/pakfire-builder build python3-build/python3-build.nm > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/usr/lib/libthread_db.so.1". > [New Thread 0x7ffff55276c0 (LWP 61317)] > > Thread 2 "pakfire-builder" received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7ffff55276c0 (LWP 61317)] > 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 > (gdb) bt > #0 0x00007ffff7fd4fa4 in ?? () from /lib64/ld-linux-x86-64.so.2 > #1 0x00007ffff7fd2544 in ?? () from /lib64/ld-linux-x86-64.so.2 > #2 0x00007ffff7fcc715 in ?? () from /lib64/ld-linux-x86-64.so.2 > #3 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 > #4 0x00007ffff7fccb75 in ?? () from /lib64/ld-linux-x86-64.so.2 > #5 0x00007ffff7fd60b1 in ?? () from /lib64/ld-linux-x86-64.so.2 > #6 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 > #7 0x00007ffff7fd581a in ?? () from /lib64/ld-linux-x86-64.so.2 > #8 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 > #9 0x00007ffff7fd5bec in ?? () from /lib64/ld-linux-x86-64.so.2 > #10 0x00007ffff7ea58a1 in ?? () from /usr/lib/libc.so.6 > #11 0x00007ffff7fcb4e1 in _dl_catch_exception () from /lib64/ld-linux-x86-64.so.2 > #12 0x00007ffff7fcb603 in ?? () from /lib64/ld-linux-x86-64.so.2 > #13 0x00007ffff7ea5811 in ?? () from /usr/lib/libc.so.6 > #14 0x00007ffff7ea5a4f in ?? () from /usr/lib/libc.so.6 > #15 0x00007ffff7e8d857 in ?? () from /usr/lib/libc.so.6 > #16 0x00007ffff7e8dc3d in ?? () from /usr/lib/libc.so.6 > #17 0x00007ffff7e8bd58 in __nss_next2 () from /usr/lib/libc.so.6 > #18 0x00007ffff7e72805 in gethostbyname2_r () from /usr/lib/libc.so.6 > #19 0x00007ffff7e33883 in getaddrinfo () from /usr/lib/libc.so.6 > #20 0x00007ffff7b7e2c9 in ?? () from /usr/lib/libcurl.so.4 > #21 0x00007ffff7b813bc in ?? () from /usr/lib/libcurl.so.4 > #22 0x00007ffff7ddd9eb in ?? () from /usr/lib/libc.so.6 > #23 0x00007ffff7e617cc in ?? () from /usr/lib/libc.so.6 > (gdb) > > > I see libcurl and libc present there so that looks to be what you indicated. > > I have curl-8.3.0 and glibc-2.38 on my system. > > Regards, > > Adolf. > > On 01/10/2023 14:51, Adolf Belka wrote: >> Hi Michael, >> >> On 01/10/2023 14:22, Michael Tremer wrote: >>> Hello, >>> >>>> On 30 Sep 2023, at 13:39, Adolf Belka adolf.belka@ipfire.org wrote: >>>> >>>> Hi Michael, >>>> >>>> I just ran git pull which took me to commit >>>> >>>> build: Add implicit dist() when a makefile is passed -- followed by make and make install. >>>> >>>> Running pakfire-builder build beep/beep.nm gave the error message "Segmentation fault" >>> >>> I can confirm this, but it seems to be a crash in libcurl/glibc: >>> >>> >>> >>>> Rolling back to >>>> >>>> cli: pakfire-builder: Implement "image create" >>>> >>>> then worked with beep fine. >>> >>> However, if you say rolling back works, then we must trigger the crash in Pakfire somehow, because libcurl/glibc stays the same. >> My earlier rolling back worked. However when I then tried to create a new package, python3-build, it then segfaulted again. >> >> Then I rolled back a long way and tried again with the python3-build and again it segfaulted. >> >> So it looks like the segfault is not always consistent but rolling back did not actually help as there are still packages that end up with a segfault. >> >> Regards, >> Adolf. >>> >>> In IPFire we use c-ares as a resolver for cURL which does not crash like this. >>> >>>> Then tried git pull, make, make install again and this time beep built without any problems. Then I tried sqlite and I got the segmentation fault again. >>> >>> This is very early when Pakfire is starting up and refreshing its repository information. >>> >>>> Then I rolled back and sqlite built okay. Then ran sqlite again and this time part way through building my laptop logged out from my session. >>>> >>>> Logged back in and ran sqlite again and this after a short while everything froze and I had to reset the laptop. >>>> >>>> So rolling back to >>>> >>>> cli: pakfire-builder: Implement "image create" >>>> >>>> Does not stop the problem happening it just seems to show itself in different forms. >>>> >>>> I have now rolled back to >>>> >>>> cli: pakfire: Implement --yes >>>> >>>> which I think is where my last git pull had placed me. >>>> >>>> Running pakfire-builder on beep twice and sqlite four times has given no problems so that commit stage is confirmed good. >>>> >>>> Let me know if there is anything else I should try to narrow the problem down. >>> >>> Good question. I have a VM with archlinux where I can reproduce the crash. I will have a look what I can do to fix this… >>> >>> -Michael >>> >>>> >>>> Regards, >>>> Adolf. >>>> >>>> >>>> -- >>>> Sent from my laptop >>>> >>> >> >
-- Sent from my laptop
-- Sent from my laptop
-- Sent from my laptop
-
Adolf Belka -
Michael Tremer -
Stefan Schantl