backtrace printing gcc, tos programs

[saulot]

Hi,
anyone knows if there is something like backtrace printing library for TOS programs written in c? I would like to dump something like this in my debug builds on crash, but I don't know if I can use something existing (maybe someone has done it before) or roll out my own.

[mfro]

Unfortunately, no. Sadly, we don't have a usable gdb.

The best I've been able to come up with is: let the compiler/linker write a mapfile (-Wl,-map -Wl,mapfile) and do your test runs using EmuTOS or MiNT.
Both will provide a crash address if the programs fail. You can then cross check the map file and at least find the function that failed.

Provided you have compiled using -g, m68k-atari-mint-addr2line will also bring up the failed source line if you have the crash address.

[saulot]

Thanks, I've suspected it. Oh well...

[Eero Tamminen]

Build your program with debug symbols, run it under Hatari from GEMDOS HD, drop to debugger to enable profiling ("profile on"), or enable it at start, and tell Hatari to invoke debugger on crash, and you'll see backtrace.

(Hatari generates backtrace automatically on entering debugger, when profiling has been enabled with debug symbols being present.)

See Hatari debugger manual for details: https://hatari.tuxfamily.org/doc/debugger.html

[saulot]

Thanks, I will try it.. But I will need something like this in native programs anyway at some point.

[Eero Tamminen]

It's a bug in Hatari emulation if crash that happens with real HW does not happen also under emulation (when HW and SW configs match).

If you need to know all functions called by your program, you can also ask Hatari to trace symbols: "--trace cpu_symbols,dsp_symbols".

[mikro]

Another option is to bite the bullet and compile the program with gcc2 + use the old gdb.

[saulot]

It will not happen, because I'm using C99> ...

[cdpjenkins]

Just curious, can anyone tell me what it was that caused gdb to not work any more? Given mikro's comment above, I assume it used to work back in the days of gcc 2 and an older gdb...

[mikro]

Vincent (@BlankVector here) could tell you more but in short, a.out format (yeah, that one which is about to be removed from the Linux kernel these days) has been gradually let rotting, so with every new gdb version things got more and more obsolete/untested/broken to the point where the most basic thing - breakpoints - stopped working.

The solution is obviously to switch into ELF (as the rest of the world) but it brings its own challenges (mainly for compatibility in plain TOS). There has been a few attempts in this direction but nothing definite.

[BlankVector]

IIRC there is a bug in GCC 4 when it generates STABS debug information. This is the format of debug information used in old a.out binaries. I never investigated what really happens in GCC, and GNU people don't care about a.out stuff anymore. Maybe that's just a misconfiguration, or a more serious issue, I have no idea. But that could also be something very simple, who knows.

[saulot]

@eero I've launched my app as you said, symbols are loaded, I've got a crash, but I don't have any meaningful backtrace. I've tried disabling optimisations, dri/gst symbols are generated, gcc symbols are also generated(-g and -grecord-gcc-switches).

Hatari(latest automated build from antartctica.no) was launched with: '--natfeats true --trace cpu_symbols,gemdos,xbios,natfeats'. Enabled debugger on exceptions with profiling enabled:
ralt+pause, setopt -D, profile on, cont.
After that I've launched an app from desktop and it crashes(as expected):

HatariBacktraceTest.png

But still I don't recognize this place. Symbols are loaded for sure, because I can list them via symbols command.
I've also have natfeats issue, logging doesn't work for some reason for me (my logger can output to serial, console etc and it works on real hardware, I cannot see anything in Hatari console related to natfeats). I've enabled natfeats via switch and I've got it enabled in cfg file (bNatFeats = TRUE).

[Eero Tamminen]

After that I've launched an app from desktop and it crashes(as expected):
HatariBacktraceTest.png
But still I don't recognize this place. Symbols are loaded for sure, because I can list them via symbols command.

Profiler tracks call hierarchy through locations which have a symbol. Backtrace is composed of symbols through which the code ran since previous debugger invocation, and which have not been returned yet.

But you haven't loaded the symbols, as I can see them getting loaded only in the screenshot.

[Eero Tamminen]

I've also have natfeats issue, logging doesn't work for some reason for me (my logger can output to serial, console etc and it works on real hardware, I cannot see anything in Hatari console related to natfeats). I've enabled natfeats via switch and I've got it enabled in cfg file (bNatFeats = TRUE).

On which OS / platform this is / are you having problems with Hatari console output in general, or just NatFeats?

Does e.g. the "nf_ahcc.tos" test program coming with Hatari work: https://git.tuxfamily.org/hatari/hatari ... /natfeats/ ?

(It uses NatFeats to query + output Hatari version and invoke Hatari debugger. Or if NatFeats is not enabled, complain about that.)

[saulot]

@Eeero I'm on Windows 10 pro, 64bit 212H. Just downloaded latest Hatari build(2.4.0 June 18th), release and launched NF_AHCC.TOS. I deleted old config and generated new ones to be 100% sure that problem isn't configuration related.
Hatari was launched with: --natfeats yes --trace cpu_symbols,gemdos,xbios,natfeats. Here are logs what is going on after NF_AHCC.TOS launch:

Hatari_nf_ahccTosTest.png

It doesn't invoke Hatari debugger just closes itself. I don't see anything natfeats related in output, even on startup.
Config is set for Falcon030 with TOS 4.04(030,16mhz,14mib ram, dsp on).
I've noticed bug that if console window is not enabled in base config say (hatari.cfg) and you have new config with another name loaded via gui with console enabled (bConsoleEnabled TRUE), then after new configuration load(with bConsoleEnabled TRUE), console window will not open.

[Eero Tamminen]

@Eeero I'm on Windows 10 pro, 64bit 212H.
...
I've noticed bug that if console window is not enabled in base config say (hatari.cfg) and you have new config with another name loaded via gui with console enabled (bConsoleEnabled TRUE), then after new configuration load(with bConsoleEnabled TRUE), console window will not open.

Dang, I was afraid it was Windows. None of us has Windows (Windows binaries are cross-compiled on Linux), so anything Windows related needs to be contributed (coded, debugged and tested) by Hatari Windows users.

Here's the totality of Hatari Windows console support (contributed well over decade ago): https://git.tuxfamily.org/hatari/hatari ... c/gui-win/

It's only called from main() on startup, so it's no wonder if changing it later does not work properly (I added the -W option to enabled it in 2013, before that it was enabled unconditionally).

If trace output is visible on console, NatFeats NF_STDERR output should show there too though, as it's just writing to Hatari process stderr: https://git.tuxfamily.org/hatari/hatari ... ats.c#n130

I do not understand how running things under Windows could break something like that...

Maybe you could install WSL2 + Ubuntu from Windows store, do "sudo apt install hatari" under that, and try it there?

[czietz]

Certainly works for me. Tested under Windows with hatari-win64-ce8058b.zip from http://antarctica.no/~hatari/latest/windows/.

hatari-natfeats.png

[Eero Tamminen]

Hatari was launched with: --natfeats yes --trace cpu_symbols,gemdos,xbios,natfeats. Here are logs what is going on after NF_AHCC.TOS launch:

If you use "hatari -W --log-level debug --trace os_base --natfeats yes nf_ahcc.tos", which of the following (expected) outputs you see:

Code: Select all

DEBUG: Native Features enabled.
DEBUG: ARG = autostart program: nf_ahcc.tos
...
GEMDOS 0x3D Fopen("C:\EMUDESK.INF", read-only) at PC=0xE4F9CC
DEBUG: Virtual INF file 'C:\EMUDESK.INF' matched.   <== when using EmuTOS
-> FD 64 (read-only -> read-only)
GEMDOS 0x3E Fclose(64) at PC 0xE4F9CC
DEBUG: Virtual INF file removed.
...
GEMDOS 0x4B Pexec(0, "C:\NF_AHCC.TOS", [0]"", 0x14964) at PC 0xE64CBA
GEMDOS 0x4B Pexec(7, 0x7, 0x149f0, 0x14964) at PC 0xFA003A
Emulator name:                                     <== NF_STDERR output from nf_ahcc.tos
Hatari v2.4.0-devel (Jul  1 2022)
Invoking debugger...