[A]tari [G]ame [T]ools - 2D prototyping engine for STE

GFA, ASM, STOS, ...

Moderators: simonsunnyboy, Mug UK, Zorro 2, Moderator Team

Post Reply
User avatar
prog99
Captain Atari
Captain Atari
Posts: 159
Joined: Thu Jun 19, 2003 8:08 pm
Location: Ross & Cromarty
Contact:

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by prog99 »

Playmobil wrote: Sat Sep 03, 2022 4:35 pm Thanks @prog99 & @dml !
I’m away at the moment but will refresh my image with the latest build and make it public when I get back.
All my real skills are undervalued
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2203
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by TheNameOfTheGame »

dml wrote: Sun Sep 04, 2022 12:06 pm I have pushed out a bunch of improvements, mainly to the sprite system. Some of it optimizations and some are consistency fixes.

- EMX & EMH sprite generator improvements, drawing is slightly quicker
- optimization to cliptesting on all paths
- hitflash/drawflags is now observed on all (non-fastpath) sprite paths (e.g. EMXSPR uses it, EMXSPRQ ignores it for slight speed gain on abundant objects like bullets, particles etc.).
- drawflags encoding now supports blinking, strobe-fill (both with decay counter) and invisible mode on these paths. the state for this is per-entity-instance so drawing the same sprite for different entities with different drawflags is ok.
- SLAB (and all other paths) drawing now works with x-guardband clipping. previously SLABs would turn invisible if they touched the left viewport edge.
- fixed bug in SLR custom clear shapes not always working correctly with guardband clipping
- EMX & EMH sprite generator annealing is now multithreaded (it uses 3/4 of available cores while annealing)
- more detailed stats on generated EMX/EMH sprite frames, to help optimize settings for less memory or fewer cycles

Some improvements to PCS also (mainly bugfixes and threadpool now used when generating dualfield & superpalettes).

There are some more optimizations on the way, mainly for EMH but they will probably not surface until next weekend or later.
Wow, good work there! SLAB drawing improvements with x-guardband are nice to see.
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

Have added a new project: examples/sprtest

The main purpose is to test new sprites and compare the same sprite in different drawing formats with various optimizations and settings. It is not meant for exact performance profiling of anything. It is only provided to make quick comparisons and pick a format. (I would use a different approach to profile the drawing functions more exactly, using batches).

It includes keyboard control to toggle the 4 formats on/off and move them around independently. It includes a scanline timer which shows approx drawing and clearing times (for default clearing method, although clearing is a configurable thing).

The assets script also makes copies of the final files and packs them to show relative compression gains.

To regenerate the sprites, build and prepare the test, invoke this command:

Code: Select all

./assets_com.sh && make remote && ls -l assets/*.???_
It should list the final file sizes, including the compressed copies, to compare e.g.:

uncompressed:

Code: Select all

24708 Sep  4 15:48 1ems.ems
27444 Sep  4 15:48 2emx.emx
16082 Sep  4 15:48 3emh.emh
 952 Sep  4 15:48 4slab.slr
5294 Sep  4 15:48 4slab.sls
compressed:

Code: Select all

4896 Sep  4 15:23 assets/1ems.ems_
7237 Sep  4 15:23 assets/2emx.emx_
4946 Sep  4 15:23 assets/3emh.emh_
139 Sep  4 15:23 assets/4slab.slr_
3434 Sep  4 15:23 assets/4slab.sls_

snap.png

When run, the project shows 3 scanline timings in the AGT console area:

TOT: total time from first scan, including engine overheads (this is always more than the DRW+CLR time)
DRW: just the drawing time, but includes engine overhead for the drawing pass
CLR: just the clearing time, same deal

When no objects are drawn, it will try to calibrate-out fixed engine overheads from DRW and CLR but again, it is just approximate.

To set up your own sprite to test, just change these lines:

TEST=${SRC}/bug80w.png
TESTCUT="-ackeyrgb FF:FF:00"

The first is the sprite sourcedata/image, which should be in source_assets/

The second is the cutting command to match the image. There are 3 different ways to cut sprites but -ackeyrgb (auto cut from crosshair with keyed rgb) is probably the most lazy / easiest to try from a paint package. The others involve specific cutting commands or using a sprite guide script.


Oh yeah - and the scan timer (and rasters of course) only make sense if everything draws in <=1 VBL. If it overruns, the console will print errors and ask you to toggle some objects to get it back under 1VBL.
You do not have the required permissions to view the files attached to this post.
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

prog99 wrote: Sun Sep 04, 2022 1:11 pm I’m away at the moment but will refresh my image with the latest build and make it public when I get back.
Great, thanks!
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

TheNameOfTheGame wrote: Sun Sep 04, 2022 1:46 pm Wow, good work there! SLAB drawing improvements with x-guardband are nice to see.
Thanks. I'll probably work on y-guardband next, as it would remove the need to store a lot of yclip info in the EMX format (it can be overridden with -nc flag but still annoying) which bloats out the files, and is otherwise necessary for EMX when it crosses the top/bottom viewport edge, otherwise they turn invisible. I think its the final annoying restriction that makes the slower EMS format still preferred over EMX/EMH in too many scenarios. With that solved, the EMS format (and EMX without -nc) will mainly be valuable for mid-screen parallax splits, which do require exact y-clipping to cross playfield edges efficiently.
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

prog99 wrote: Sun Sep 04, 2022 1:11 pm
Playmobil wrote: Sat Sep 03, 2022 4:35 pm Thanks @prog99 & @dml !
I’m away at the moment but will refresh my image with the latest build and make it public when I get back.
Yes ! Thanks !
User avatar
prog99
Captain Atari
Captain Atari
Posts: 159
Joined: Thu Jun 19, 2003 8:08 pm
Location: Ross & Cromarty
Contact:

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by prog99 »

Playmobil wrote: Sun Sep 04, 2022 4:47 pm
prog99 wrote: Sun Sep 04, 2022 1:11 pm
Playmobil wrote: Sat Sep 03, 2022 4:35 pm Thanks @prog99 & @dml !
I’m away at the moment but will refresh my image with the latest build and make it public when I get back.
Yes ! Thanks !
Ok, give this a whirl in your source folder. There are a few caveats as per the read me (https://hub.docker.com/repository/docke ... /agt-tools)
docker run -v $(pwd):/agt-tools/build -it prog99/agt-tools
Thats for macOS & linux and other x86 unix-like os's. I've no access to a windows environment where I can run docker to test.

If you have a more complicated structured then you'd have to do some tinkering with how you mount the volume and run your build. I've just done the simplest use case.
All my real skills are undervalued
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

prog99 wrote: Sun Sep 04, 2022 9:58 pm Ok, give this a whirl in your source folder. There are a few caveats as per the read me (https://hub.docker.com/repository/docke ... /agt-tools)
docker run -v $(pwd):/agt-tools/build -it prog99/agt-tools
Thats for macOS & linux and other x86 unix-like os's. I've no access to a windows environment where I can run docker to test.

If you have a more complicated structured then you'd have to do some tinkering with how you mount the volume and run your build. I've just done the simplest use case.
OK ! I've done this steps :

Step 1 : Create an account on docker.com and downloaded Docker Desktop for Windows
Step 2 : Installation of Docker Desktop (It complain about WSL 2)

Next Steps tomorrow : Installing WSL2, and learn & learn & learn about how using Docker !

Many Thanks @prog99
User avatar
prog99
Captain Atari
Captain Atari
Posts: 159
Joined: Thu Jun 19, 2003 8:08 pm
Location: Ross & Cromarty
Contact:

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by prog99 »

If you need to discuss offline then just drop me a pm and we can sort out the finer details without going off topic on AGT. Unfortunately the one windows machine I do have access to is a work one and whilst docker is ok with our internal repos docker hub is strictly verboten!
All my real skills are undervalued
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

For some reason I'm not getting notifications here anymore. Not that it's a big deal, but I might be slow to answer stuff...
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

Have pushed out latest changes from recent evenings experimenting & some crazy debugging sessions (!). I had most of it working last night but there was at least one bug and I wanted to review all the edits before commiting anyway - a lot of files were hit.

In summary - all sprites can cross all screen borders safely in all scroll configurations. It is now possible to pick and choose sprite formats for speed / memory, without also being cornered with tricky clipping restrictions - which most of the primitives had to some degree or other. This is implemented through configurable X & Y guardbands - hidden drawing area around the screen.

Physical clipping is still supported by some of the primitives but with a guardband active none of them rely on this. I'll document the detailed differences on the wiki soon.

The guardband itself isn't the hard part - this is easy enough on STE. Making it work with *everything else* in every possible configuration was the hard part.

This now works for all sizes of sprite and all types, up to screen size. It also works with all clearing methods, including custom SLR clearing outlines (full and edges-only) and embedded occlusion maps (tiles refreshed around sprite edges for very big sprites - a debugging nightmare at my side :-p ). It works with the playfield in y-loopback mode and/or with tilerestore shadow buffer enabled.


Guardbands are configurable globally and y-guard can be different per playfield. x-guard can also be set per playfield but i think it affects some of the 68k map animation code so for now its best to pick one value and stick with it across all playfields. EMX code-generated sprites need to specify the x-guard value at build time anyway so another good reason to pick one value for a project, to avoid confusion at least.



Other improvements, optimizations mixed in, including some optimizer improvements to the cutting tool.

There are some other AGT things I'm still looking at but will probably take a break from this for a while, unless something turns up broken.
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2203
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by TheNameOfTheGame »

Wow, another great update! Thanks for all these improvements.

Just some feedback that I am seeing a problem compiling some of the examples with the new push. The ones giving problems are:
hiworld
map8x8
map8x8x2
map16x16x2layer
parallax

The errors are mostly the same as this:

Code: Select all

build/aglib_ste.a(pflib.o):pflib.o:(.text+0x6d2e): undefined reference to `_drawcontext'
build/aglib_ste.a(pflib.o):pflib.o:(.text+0x6d4c): undefined reference to `_drawcontext'
build/aglib_ste.a(pflib.o):pflib.o:(.text+0x6d90): undefined reference to `_drawcontext'
build/aglib_ste.a(pflib.o):pflib.o:(.text+0x6dae): undefined reference to `_drawcontext'
collect2: ld returned 1 exit status
make: *** [../../makerules.mintgcc:10: build/map8x8.tos] Error 1
sprtest is giving a different error:

Code: Select all

sprtest.cpp: In function 'int AGT_EntryPoint()':
sprtest.cpp:541:24: error: 'drawcontext' was not declared in this scope
make: *** [../../makerules:169: build/sprtest.o] Error 1
Any suggestion to get them to compile?

*Edit* Another issue popped up. The example "giantsprites" does compile, but when I try to run it I get this error:

giantspriteserror.png
You do not have the required permissions to view the files attached to this post.
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

The missing symbol is something that was removed from the engine, but added to the samples. I'm pretty sure the changes made it into the repo as I have none outstanding - but I'll double check it.

Try cleaning the project fully to get it to rebuild all the sample & engine code again. The sprites also need rebuilt because the version info changed.

Code: Select all

> make clean-all && make disk1
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

I have built examples/parallax with gcc 4.6.4 on a PC and built examples/map8x8 with gcc 7.5.0 on the RPI (pulled from git) and don't see any link errors so I presume these are just stale objects needing cleaned from the last version.

For the last error loading spritesheets (screenshot), that's exactly what it is signalling. The clean-all rule should zap all the data and code back to a clean slate.
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2203
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by TheNameOfTheGame »

dml wrote: Fri Sep 09, 2022 6:33 pm The missing symbol is something that was removed from the engine, but added to the samples. I'm pretty sure the changes made it into the repo as I have none outstanding - but I'll double check it.

Try cleaning the project fully to get it to rebuild all the sample & engine code again. The sprites also need rebuilt because the version info changed.

Code: Select all

> make clean-all && make disk1
Thanks, that "make clean-all && make disk1" worked for all but the sprtest which give this error.

Code: Select all

sprtest.cpp: In function 'int AGT_EntryPoint()':
sprtest.cpp:541:24: error: 'drawcontext' was not declared in this scope
make: *** [../../makerules:169: build/sprtest.o] Error 1
Btw, assets_com.sh is not executable for sprtest as pulled from github. Had to "chmod +x" on it.

But running assets_com.sh for sprtest gives an error:

Code: Select all

extracting unique tiles...
emitted tilemap [assets/bg.ccm]
emitted map visualisation [assets/bg_mapvis.tga]
emitted map index visualisation [assets/bg_index.tga]
...done
final dictionary contains 116 unique tiles with footprint @ 0.127193 of original image.
 tile reuse seems ok but keep an eye on things...
converting 116 8-bit tiles into planar format...
...done
CRC=8df312f9 w=7442
emitted tiles [assets/bg.cct]
[A]tari [G]ame [T]ools: playfield/sprite/slab cutter v0.474 / dml
preshift step=1 scale=0
key colour from rgb: r(ff):g(0):b(ff):
error: invalid argument: -mpl
press any key...
Last edited by TheNameOfTheGame on Fri Sep 09, 2022 7:32 pm, edited 2 times in total.
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

I'm able to built some tutos/exemples now with the help of @prog99 and his Docker ! Thanks to him !

But all exemples wich contain assets give me an error...
It complain about the line with :
set -e
in the "assets_com.sh"

Error is Invalid command... So PRG build is OK, but I have no Assets (EMX, EMS, etc etc)
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

Ok the sprtest example is probably broken at my end. I'll look at that.

The scripts - I guess Windows stripped them (grr) but I'll fix that first.
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

Playmobil wrote: Fri Sep 09, 2022 7:28 pm But all exemples wich contain assets give me an error...
It complain about the line with :
set -e
in the "assets_com.sh"
Error is Invalid command... So PRG build is OK, but I have no Assets (EMX, EMS, etc etc)
What shell are you using? The 'set -e' should cause the script to exit on an error. It's not critical but I had assumed that was already working on various shells.
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

dml wrote: Fri Sep 09, 2022 7:35 pm What shell are you using? The 'set -e' should cause the script to exit on an error. It's not critical but I had assumed that was already working on various shells.
I'm under Windows 10, and using PowerShell in Administrator...

Give me that :

Code: Select all

PS C:\test2> docker image ls
REPOSITORY         TAG       IMAGE ID       CREATED      SIZE
prog99/agt-tools   latest    49dcadb05562   5 days ago   250MB
PS C:\test2> docker run -v ${PWD}:/agt-tools/build -it prog99/agt-tools
BUILDING COMMON ASSETS: giantspr
./assets_com.sh
./assets_com.sh: 5: set: Illegal option -
make: *** [Makefile:193: cache/assets_common.tag] Error 2
PS C:\test2>
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

if I remove SET -e then it give me that :

Code: Select all

PS C:\test2> docker run -v ${PWD}:/agt-tools/build -it prog99/agt-tools
BUILDING COMMON ASSETS: giantspr
./assets_com.sh
: not foundm.sh: 7:
: not foundm.sh: 9:
: not foundm.sh: 13:
: not foundm.sh: 16:
: not foundm.sh: 19:
configured AGT native tools @ /agt-tools/agtools/bin/Linux/x86_64
: not foundm.sh: 23:
: not foundm.sh: 27:
: not foundm.sh: 29:
: not foundm.sh: 30:
: not foundm.sh: 34:
: not foundm.sh: 37:
: not foundm.sh: 40:
: not foundm.sh: 44:
/agtcut: not found6: /agt-tools/agtools/bin/Linux/x86_64
: not foundm.sh: 47:
: not foundm.sh: 50: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 51: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 52:
: not foundm.sh: 53:
: not foundm.sh: 57:
: not foundm.sh: 59:
: not foundm.sh: 60:
: not foundm.sh: 63:
: not foundm.sh: 66:
: not foundm.sh: 70:
: not foundm.sh: 74:
: not foundm.sh: 77:
: not foundm.sh: 81:
/agtcut: not found5: /agt-tools/agtools/bin/Linux/x86_64
: not foundm.sh: 86:
/agtcut: not found0: /agt-tools/agtools/bin/Linux/x86_64
/agtcut: not found1: /agt-tools/agtools/bin/Linux/x86_64
: not foundm.sh: 92:
/agtcut: not found6: /agt-tools/agtools/bin/Linux/x86_64
/agtcut: not found7: /agt-tools/agtools/bin/Linux/x86_64
: not foundm.sh: 98:
: not foundm.sh: 99:
: not foundm.sh: 102: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 103: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 104: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 105: /agt-tools/agtools/scripts/pack.sh
: not foundm.sh: 106: /agt-tools/agtools/scripts/pack.sh
make: *** [Makefile:193: cache/assets_common.tag] Error 127
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

That's not a build environment tested with AGT... Is that the environment provided in the docker image?
User avatar
TheNameOfTheGame
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2203
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by TheNameOfTheGame »

The giantsprites example looking great! Could never imagine something like that running on a STE.
Playmobil
Captain Atari
Captain Atari
Posts: 283
Joined: Fri Nov 13, 2015 7:40 pm

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by Playmobil »

dml wrote: Fri Sep 09, 2022 7:52 pm That's not a build environment tested with AGT... Is that the environment provided in the docker image?
Yes ! From prog99, some posts above...
prog99 wrote: Sun Sep 04, 2022 9:58 pm Ok, give this a whirl in your source folder. There are a few caveats as per the read me (https://hub.docker.com/repository/docke ... /agt-tools)
docker run -v $(pwd):/agt-tools/build -it prog99/agt-tools
Thats for macOS & linux and other x86 unix-like os's. I've no access to a windows environment where I can run docker to test.

If you have a more complicated structured then you'd have to do some tinkering with how you mount the volume and run your build. I've just done the simplest use case.
User avatar
dml
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 3951
Joined: Sat Jun 30, 2012 9:33 am

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by dml »

I found a new issue with sprtest sample (and which is in the engine itself) but I think I'm done for the night - I'll follow up with a fix for that next time :)
User avatar
prog99
Captain Atari
Captain Atari
Posts: 159
Joined: Thu Jun 19, 2003 8:08 pm
Location: Ross & Cromarty
Contact:

Re: [A]tari [G]ame [T]ools - 2D prototyping engine for STE

Post by prog99 »

Playmobil wrote: Fri Sep 09, 2022 8:01 pm
dml wrote: Fri Sep 09, 2022 7:52 pm That's not a build environment tested with AGT... Is that the environment provided in the docker image?
Yes ! From prog99, some posts above...
prog99 wrote: Sun Sep 04, 2022 9:58 pm Ok, give this a whirl in your source folder. There are a few caveats as per the read me (https://hub.docker.com/repository/docke ... /agt-tools)
docker run -v $(pwd):/agt-tools/build -it prog99/agt-tools
Thats for macOS & linux and other x86 unix-like os's. I've no access to a windows environment where I can run docker to test.

If you have a more complicated structured then you'd have to do some tinkering with how you mount the volume and run your build. I've just done the simplest use case.
Edit assets_com.sh and remove the line that sets AGTROOT. Then try again. The docker image sets this for you.

The image defaults to running make but you can also run other commands too such as -
docker run -v ${PWD}:/agt-tools/build -it prog99/agt-tools make clean
or if you fancy a poke around in the shell
docker run -v ${PWD}:/agt-tools/build -it prog99/agt-tools bash
etc...

Just refreshing the image with the latest changes.
All my real skills are undervalued
Post Reply

Return to “Coding”