How to determine if returned address is fastram

GFA, ASM, STOS, ...

Moderators: simonsunnyboy, Mug UK, Zorro 2, Moderator Team

User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

How to determine if returned address is fastram

Post by saulot »

Hi, What is the best method of determining if allocated memory address returned via MxAlloc() belongs to ST or TTRam?
Can I just assume that ST ram is always below 16MiB adress space or this is more complicated? I need to determine type of memory if I pass PREFER_ST/TT flags (some machines can have fast ram or not).

Thanks,
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: How to determine if returned address is fastram

Post by dml »

I think TT/fast ram always must be > 24bit/16meg range otherwise there are conflicts with the HW register/cartridge areas and with the STRam address range in general on Ataris.

On 030+ machines it is still possible to deliberately MMU-map the fast ram (from above 16mb) to somewhere below that - e.g. between 4mb...14mb - but that would be asking for compatibility trouble :) so I doubt it ever happens. Even if a VMEM driver like Outside tries to use TTRam, it can't mis-represent it as pages of STRam in case something tries to DMA from there. (It could however mis-represent STRam as TTRam without necessarily causing problems).

An slightly more clean way could involve looking at the rambase system variables and figuring out if the address lies in the TOS-published TTRam window but it's hard to say if that actually is any better than the >16mb test. Doing both checks might find any outliers / weird machine configs.
ThorstenOtto
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2367
Joined: Sun Aug 03, 2014 5:54 pm

Re: How to determine if returned address is fastram

Post by ThorstenOtto »

The question is: why do you need to know that? If it is for determining whether that memory is usable as video memory or DMA, then the <16MB check is the right thing, since the only reason why fastram can't be used for that is the limited address space of DMA/shifter. If you need to know whether that memory is actually faster than ST ram, then that check may not tell you the truth. There are hardware extensions available that return memory addresses above 4MB, but below 16MB via Mxalloc().

Also don't forget that you can still get ST ram via call to Mxalloc(PREFER_TT_RAM), when TT-ram is generally available, but not enough to fulfill your request.
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2834
Joined: Sun Jul 31, 2011 1:11 pm

Re: How to determine if returned address is fastram

Post by Eero Tamminen »

Area above 10MB is for VME HW address space, if machine includes VME, and area above 14MB (until 16MB limit) is not RAM on any machine.
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

@ThorstenOtto: I wrote heap smashing detector for stram, ttram memory (something similar to Windows CRT one). And I want to be sure that returned address is either from st ram or ttram, because of reason you wrote about. If I set PREFER_TT on st without fast ram I will receive an address from ST ram if block in ST ram is available. It's just to determine to which heap info structure (st or tt ram or video ram(in case of sv or ctpci)) given allocation needs to be stored.

So, probably I will need to check it on several configurations.. And yes vme is a special case and anything between 14mib and 16MiB is inaccessible.. And yes, i thought about looking into system variables for a moment(ramtop, memtop and friends), but I thought that maybe I will ask around first.
Anyone knows maybe how exactly MxAlloc() works and how it stores information internally about allocated memory blocks?
ThorstenOtto
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2367
Joined: Sun Aug 03, 2014 5:54 pm

Re: How to determine if returned address is fastram

Post by ThorstenOtto »

I still don't get the idea. Your programs heap (if that is what you are talking about) may be in a different area than Malloced() ones. So you just have to handle that, i can't see a reason to explicitely know whether it is STram or TTram.

About how Mxalloc works: that depends on the OS. EmuTOS uses two different memory pools for example. TOS 2.x and 3.x mark the addresses for TTram internally by setting the least significant bit (but you you will only see that when manually walking through the list of memory descriptors that are linked into the system variable at 0x48e). MagiC is also quite different.
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

@ThorstenOtto: I have overriden allocation routines. I can switch between malloc()/free() and Malloc()/MxAlloc()/MFree() variants (with simple define). I have one interface, but I can switch allocation routines anytime depending on my needs. I assume I don't have a control over what kind of flag is passed to MxAlloc() from user program under TOS. When I allocate memory I insert some info about memory block and some guards with predefined value before and after requested memory block and I'm adding it to internal linked list.
With basic ST_RAM/TT_RAM request, there is no problem (either I get address from given adress range or not(0/NULL)). But I want to recognise what kind of memory is returned from MxAlloc(), when PREFER_FLAG is used to determine to which linked list I have to attach the new block (I have three linked lists/heads: stram, ttram, video ram (for ctpci or supervidel)). So, I don't want to insert info about new memory block to TT ram/fast one, when I passed PREFER_TT on Falcon without TT RAM (just for example, because there are other variants certainly).
I remove blocks from adequate list on memory free/release.

I have check functions which determine if given block guards are intact (before/after requested memory region) and I can detect leaks for example by going through the lists on program end (for example). Additionally I set values of allocated memory to know values on allocation/release. All of this is of course only in debug builds.
With libc malloc()/free() there is no problem, there is one list. With ST/TT ram/video ram I want three. But this video ram one is still puzzling me, because it's based on ct60_vmalloc()(both are supposed to be used on SV/ctpci), but I'm not sure how this memory is managed(if at all).
joska
Hardware Guru
Hardware Guru
Posts: 5342
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: How to determine if returned address is fastram

Post by joska »

saulot wrote: Mon Oct 31, 2022 9:48 am Can I just assume that ST ram is always below 16MiB adress space or this is more complicated?
It is more complicated.

- On Falcon and TT all RAM below 16Mb can be accessed by the videochip and DMA.
- On Falcon and ST's with 32-bit accelerators and TT-RAM only the CPU can access RAM above 16Mb.
- On the TT I believe that SCSI DMA can access TT-RAM, in addition to the CPU.
- On 68000 ST's with altRAM-cards only RAM below 4Mb can be accessed by the videochip and DMA, but all RAM below 16Mb can be accessed by the blitter.
- Emulators might offer 14 Mb of ST-RAM which can be accessed by all hardware.
Jo Even

VanillaMiNT - Falcon060 - Milan060 - Falcon040 - MIST - Mega STE - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

Well, I'm aware to issues related to DMA/Video access, it's irrelevant at this point. I'm thinking now about simple check like determining if hardware has vme or not and check if returned address is below 14mib or 10Mib adress range and if yes then it would mean ST ram, otherwise it would be tt-ram. But probably I will have to perform some tests, I have plenty of hardware ;).
ThorstenOtto
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2367
Joined: Sun Aug 03, 2014 5:54 pm

Re: How to determine if returned address is fastram

Post by ThorstenOtto »

saulot wrote: Mon Oct 31, 2022 1:37 pm I assume I don't have a control over what kind of flag is passed to MxAlloc() from user program under TOS.
When using Mxalloc directly, of course you do have control. When using just Malloc(), TOS uses the flag from the program header, so you also have some kind of control.
I have three linked lists/heads: stram, ttram, video ram (for ctpci or supervidel)
Why not just use a single list?
I have check functions which determine if given block guards are intact (before/after requested memory region)
I've already done similar things in several projects, and i never had the problem to distinguish between ST/TT ram.
But this video ram one is still puzzling me
Video ram might be a different thing, if it is not allocated using Malloc() or Mxalloc(). But i guess that is allocated only once.
mikro
Hardware Guru
Hardware Guru
Posts: 2997
Joined: Sat Sep 10, 2005 11:11 am
Location: Kosice, Slovakia
Contact:

Re: How to determine if returned address is fastram

Post by mikro »

A really dumb way of doing the check would be:

old_size_tt = Mxalloc(-1, tt ram);
old_size_st = Mxalloc(-1, st ram);

Mxalloc(your_size, prefer tt ram);

new_size_tt = Mxalloc(-1, tt ram);
new_size_st = Mxalloc(-1, st ram);

And check which one has shrunk.

No, neither interrupt nor thread safe. ;)
joska
Hardware Guru
Hardware Guru
Posts: 5342
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: How to determine if returned address is fastram

Post by joska »

saulot wrote: Mon Oct 31, 2022 2:08 pm Well, I'm aware to issues related to DMA/Video access, it's irrelevant at this point.
Not quite sure what problem you're trying to solve then. If DMA/Video is irrelevant - why even bother?
saulot wrote: Mon Oct 31, 2022 2:08 pm I'm thinking now about simple check like determining if hardware has vme or not and check if returned address is below 14mib or 10Mib adress range and if yes then it would mean ST ram, otherwise it would be tt-ram.
Yes, but no need to check for VME or not. If memory allocation returns anything else than a zero, the return value will point to actual RAM. So you only need to distinguish between below and above 16Mb.

Btw. on the TT the VME address range is FE000000-$FEFFFFFF, on the MSTE it's $FF0000-$FFFFFF.
Jo Even

VanillaMiNT - Falcon060 - Milan060 - Falcon040 - MIST - Mega STE - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64
Rustynutt
Atari God
Atari God
Posts: 1752
Joined: Wed Mar 21, 2012 7:38 am
Location: Oregon

Re: How to determine if returned address is fastram

Post by Rustynutt »

And what of 8mb boards in the Falcon expansion bus? Doesn't the driver place the ram at other than "TT" RAM locations? Know it's not true fast RAM.

Also, with NOVA installed, 2 MB of ST RAM is also off limits.
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

ThorstenOtto wrote:
saulot wrote: Mon Oct 31, 2022 1:37 pm I assume I don't have a control over what kind of flag is passed to MxAlloc() from user program under TOS.
When using Mxalloc directly, of course you do have control. When using just Malloc(), TOS uses the flag from the program header, so you also have some kind of control.
I mean that I don't have a control over what system returns in case of missing fastram, when PREFER_* is used. Under Malloc() yes, there are flags, but for some cases it will cause program errors, e.g program which calls Malloc() for video buffer or dma sound and prg memory flag is set to tt ram in program header file and fast ram is actually present on system. Yes, it might work in some programs, but for me it's a hack for older programs, which could benefit from using tt-ram and cannot be updated and recompiled.
In some cases allocating TT ram gives faster memory access, so it's advantagous to use it in some scenarios.
ThorstenOtto wrote:
I have three linked lists/heads: stram, ttram, video ram (for ctpci or supervidel)
Why not just use a single list?
Well, probably I could. But separate lists have some advantages like I don't have store info about memory block type in a linked list memory block, i don't have to check memory type on each memory info block access (for example if I want to dump info about specific memory type), if given list is empty I don't have to iterate through each element. Well, maybe I will think about this if everything fails.
ThorstenOtto wrote:
I have check functions which determine if given block guards are intact (before/after requested memory region)
I've already done similar things in several projects, and i never had the problem to distinguish between ST/TT ram.
Maybe I overcomplicate things or I just approach this differently. And maybe it's not a problem.
ThorstenOtto wrote:
But this video ram one is still puzzling me
Video ram might be a different thing, if it is not allocated using Malloc() or Mxalloc(). But i guess that is allocated only once.
Yes, probably. With video I mean Supervidel for example. It has 128mib of video memory data at 0xA1000000 (112mib accessible if i correctly understand from memory map + 16MiB of ST ram shadow probably for video screen shadowing purposes). But I haven't experimented with it enough to determine if I can just grab it all from start address or if there is some part from this address range used already by a system. Secondly if SV video memory is grabbed by a system then, is it allocated via ct60_vmalloc() or something different? (like some part of video driver just grabs/reserves begining of memory range without any memory management function calls/notyfying os). But it's not a problem right now, I save it for later. Probably I have to ask Pep/Shoggoth about it at some point.

@mikro: thanks, nice idea. I don't have to be thread safe on Atari ;-) .
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

joska wrote: Mon Oct 31, 2022 3:11 pm
saulot wrote: Mon Oct 31, 2022 2:08 pm Well, I'm aware to issues related to DMA/Video access, it's irrelevant at this point.
Not quite sure what problem you're trying to solve then. If DMA/Video is irrelevant - why even bother?
It's for debugging memory issues purposes in debug builds.I want to have info of all allocated memory blocks in separate lists for fast / st ram/... and on allocation with PREFER_* I have to determine somehow from which type of memory i've received an address to decide to which list I have to add it.
joska wrote: Mon Oct 31, 2022 3:11 pm
saulot wrote: Mon Oct 31, 2022 2:08 pm I'm thinking now about simple check like determining if hardware has vme or not and check if returned address is below 14mib or 10Mib adress range and if yes then it would mean ST ram, otherwise it would be tt-ram.
Yes, but no need to check for VME or not. If memory allocation returns anything else than a zero, the return value will point to actual RAM. So you only need to distinguish between below and above 16Mb.

Btw. on the TT the VME address range is FE000000-$FEFFFFFF, on the MSTE it's $FF0000-$FFFFFF.
Right, checking below 16MiB range can be enough. But still I have to test it on real hardware.
ThorstenOtto
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2367
Joined: Sun Aug 03, 2014 5:54 pm

Re: How to determine if returned address is fastram

Post by ThorstenOtto »

saulot wrote: Mon Oct 31, 2022 4:16 pm e.g program which calls Malloc() for video buffer or dma sound and prg memory flag is set to tt ram in program header file and fast ram is actually present on system.
Yes, that can happen with some old programs that are not aware of Mxalloc(). In that case, you have to set the program header flags to return Malloc() from ST-RAM.
I don't have to be thread safe on Atari ;-) .
Threads might not be a problem (functions like malloc() in the C-library aren't thread-safe anyway), but of course that approach can fail with any multitasking system. But otherwise, a nice and simple idea ;)
czietz
Hardware Guru
Hardware Guru
Posts: 2080
Joined: Tue May 24, 2016 6:47 pm

Re: How to determine if returned address is fastram

Post by czietz »

saulot wrote: Mon Oct 31, 2022 4:51 pm Right, checking below 16MiB range can be enough. But still I have to test it on real hardware.
Sorry if I missed it in the thread: which machines are you targeting?

For example, memory on a Storm ST, MonSTer, Magnum ST is always below 16 MiB (as the 68000 can't address more). But from the point of view of TOS it's still Alt-RAM (aka "fast" RAM), i.e., its managed in a different pool than ST RAM.
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

@czietz: I would like to determine it as accurately as possible on Atari targets with/without extra hardware. I've got MonSTer and Storm, so I guess I would like to support them too. Same with ct60 with/without sv, tt and so on with cpu/ memory expansions. I don't care about emulators or clones.. This Alt-ram is another case too, so checking below 16mib will be insufficient in some configurations. Maybe sysinfo will shed some light on memory ranges available.. Maybe I will check what phystop/memtop/membottom/ramtop reports on different hardware..
czietz
Hardware Guru
Hardware Guru
Posts: 2080
Joined: Tue May 24, 2016 6:47 pm

Re: How to determine if returned address is fastram

Post by czietz »

Checking phystop might actually be a very good idea. I almost forgot, but for the ST, there is also this gigantic expansion board that extends the ST(!!) RAM to up to 14 MiB. I would imagine that phystop then shows 14 MiB, in contrast to what happens with an Alt-RAM board.

Image
User avatar
shoggoth
Nature
Nature
Posts: 1220
Joined: Tue Aug 01, 2006 9:21 am
Location: Halmstad, Sweden
Contact:

Re: How to determine if returned address is fastram

Post by shoggoth »

Vaguely some variation of:

1. Turn off interrupts
2. XOR address using glitter
3. Examine weather or not address changed, if not
4. XOR same address again
5. Add 0x00100000 to address
6. Goto 1
Ain't no space like PeP-space.
User avatar
Cyprian
10 GOTO 10
10 GOTO 10
Posts: 2721
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: How to determine if returned address is fastram

Post by Cyprian »

shoggoth wrote: Mon Oct 31, 2022 7:23 pm Vaguely some variation of:

1. Turn off interrupts
2. XOR address using glitter
3. Examine weather or not address changed, if not
4. XOR same address again
5. Add 0x00100000 to address
6. Goto 1
that would work with Amiga but not with a real computer :)
Atari BLiTTER has a full 24bit access to the bus.
Lynx I / Mega ST 1 / 7800 / Portfolio / Lynx II / Jaguar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
DDD HDD / AT Speed C16 / TF536 / SDrive / PAK68/3 / Lynx Multi Card / LDW Super 2000 / XCA12 / SkunkBoard / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Steem SSE / Aranym / Saint
http://260ste.atari.org
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

Less blitter more glitter ;-) Not all STs have blitter, though... But might be useful anyway..

@Cyprian: Are you sure that Atari has connected all the needed wires? :D
User avatar
Cyprian
10 GOTO 10
10 GOTO 10
Posts: 2721
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: How to determine if returned address is fastram

Post by Cyprian »

tak :)

thanks to that the BLiTTER has an access to the hardware registers $Fnnnnn, it can be used in PPera's driver as a fast DMA channel for the IDE device, and MPP uses it for palette changing in multicolor mode.
Lynx I / Mega ST 1 / 7800 / Portfolio / Lynx II / Jaguar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
DDD HDD / AT Speed C16 / TF536 / SDrive / PAK68/3 / Lynx Multi Card / LDW Super 2000 / XCA12 / SkunkBoard / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Steem SSE / Aranym / Saint
http://260ste.atari.org
joska
Hardware Guru
Hardware Guru
Posts: 5342
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: How to determine if returned address is fastram

Post by joska »

saulot wrote: Mon Oct 31, 2022 4:51 pm It's for debugging memory issues purposes in debug builds.I want to have info of all allocated memory blocks in separate lists for fast / st ram/... and on allocation with PREFER_* I have to determine somehow from which type of memory i've received an address to decide to which list I have to add it.
But why is this of interest if DMA/video access is irrelevant?

There are three types of RAM available:

1. ST-RAM - accessible to all hardware, video and DMA included.
2. Alt-RAM - accessible to CPU and blitter, but not DMA and video.
3. TT-RAM - accessible to CPU only (and SCSI DMA on TT).

You can differentiate between these like this:

1. ST-RAM: All RAM below phystop.
2. Alt-RAM: All RAM above phystop but below 16Mb.
3. TT-RAM: All RAM above 16Mb.

So your approach depends on *why* you need to differentiate between these types of RAM.
Jo Even

VanillaMiNT - Falcon060 - Milan060 - Falcon040 - MIST - Mega STE - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64
User avatar
saulot
Captain Atari
Captain Atari
Posts: 347
Joined: Sat Sep 18, 2004 9:09 pm
Location: Warszawa
Contact:

Re: How to determine if returned address is fastram

Post by saulot »

@joska: I've wrote about *why* already earlier. Plain housekeeping.
Post Reply

Return to “Coding”