How to use RenderTexture2d for collision detection?

[geniot]

I'm trying to write a 2d game (remastered version of Digger from 1983 - can be played online here: https://www.futrega.org/digger/)

My question is about the field that can change (it can be eaten by the digger and hobbins, destroyed by falling bags, etc.). It is also used for collision detection to make monsters move along the eaten corridors.

So my field is:

fld.target = rl.LoadRenderTexture(SCREEN_LOGICAL_WIDTH, SCREEN_LOGICAL_HEIGHT)`

I start with:

rl.BeginTextureMode(fld.target)
rl.ClearBackground(rl.Black)

Then I draw the background in two loops:

rl.DrawTextureRec(bgTexture, sourceRect, rl.Vector2{X: float32(x), Y: float32(y)}, rl.White)

Then I draw corridors like this:

rl.DrawTexture(downBlobTexture, xp+dX, yp-15+dY, rl.White)

Then I finish:

rl.EndTextureMode()

So I draw black textures onto the field to draw tunnels. Then I draw this field texture onto the screen texture.

And it all works nicely (drawing). But now I need to make monsters move around and they need to know where black is (eaten field).
How can I do it efficiently?

On one forum I read what Ray wrote:

... retrieving Texture data is a costly process to be done in a frame-basis, why you need it? What is the use case?

This is the use-case.

I could use

Color GetImageColor(Image image, int x, int y)

But I'm dealing with textures.

Should I draw to an image and then just use this sequence on frame change?

Color *pixels = LoadImageColors(imCopy);    // Load pixel data from image (RGBA 32bit)
UpdateTexture(texture, pixels);             // Update texture with new image data
UnloadImageColors(pixels);   

as in this example textures_image_processing ?

How efficient is this?

Or should I change the field both in a texture and in a in-memory image?

[kebl3541]

The short version: don't. Ray's warning about retrieving texture data applies to your use case too. The fix is not to find a cheaper readback, it is to stop asking the GPU where the walls are.

A render texture lives in GPU memory. Anything that reads it back, LoadImageFromTexture and by extension LoadImageColors on the result, has to pull the framebuffer across the bus and force the driver to finish queued work before it can hand you the pixels. That is a pipeline stall, and it costs roughly the same whether you read one pixel or all of them. Doing it per frame, or worse per monster per frame, is the expensive pattern regardless of how you arrange it.

The deeper problem is that pixels are the wrong source of truth. Digger's field is a grid, not a bitmap. The original is laid out in tiles and tunnels are carved in whole cells. If you ask the framebuffer whether a pixel is black, you are reconstructing grid state you already had before you drew it, and reconstructing it from something lossy: a black tunnel pixel and a black background pixel are indistinguishable, and any sprite drawn over the field temporarily corrupts the answer.

So keep two things that never touch each other.

1. The authoritative field, on the CPU. A plain array, one entry per cell:

typedef enum { CELL_EARTH = 0, CELL_TUNNEL, CELL_EMERALD } CellState;
static CellState field[FIELD_ROWS][FIELD_COLS];

Digging writes to this array. Collision, monster pathing and "can I move here" all read from it. Lookups are O(1) array indexing with zero driver involvement, so you can run them for every entity every frame without thinking about it.

2. The render texture, purely for display. Keep doing exactly what you are doing now, BeginTextureMode, blit the tunnel sprite, EndTextureMode, but treat it as write only. It is a cache of what the field looks like, never a record of what it is.

Both get updated at the same moment, in the one place where digging happens:

void CarveTunnel(int row, int col)
{
    field[row][col] = CELL_TUNNEL;              // logic

    BeginTextureMode(fld.target);               // presentation
        DrawTexture(downBlobTexture, col*CELL_W, row*CELL_H, WHITE);
    EndTextureMode();
}

That also answers your last question, whether to change the field both in a texture and an in memory image. Nearly, but use a small array of your own enum rather than an Image. An Image makes you encode game state as colours and decode it back out, which is the same lossiness problem in a cheaper wrapper. A CellState grid says what you mean, is a few hundred bytes, and is trivial to serialise when you want save games or level files.

One thing you get for free once state is on the CPU: monster AI stops being per pixel. Because tunnels are carved in whole cells, a monster only needs to consider its four neighbouring cells at the moment it reaches a cell centre, which is how the original behaved and why its monsters move on that characteristic grid. Sub pixel collision testing against a bitmap would actually be less faithful to the game you are remastering.

The readback route in textures_image_processing is fine for what that example does: one off, load time, or editor side image work. It is not meant for frame basis queries, which is what Ray was getting at.

[geniot]

Thank you for your elaborated answer! I really appreciate it. Since the time when I asked the question I've done some work which can be seen here:

https://geniot.github.io/digger/ The code is here: https://github.com/geniot/digger It's still in the initial phase but the basis is laid.

So I guess I'm reverse engineering the game, not remastering it. It's less faithful to the original game. Movement and collisions are pixel perfect. But I'm using a grid for movement:
https://github.com/geniot/digger/blob/main/src/grid_move.go#L25 - it is a bidemensional array

For collision with the field I'm drawing both the texture and the image of the field (one for writing, the other for reading):
https://github.com/geniot/digger/blob/main/src/field.go#L113

I even wrote a debug function that compares the pixels of these two:
https://github.com/geniot/digger/blob/main/src/field.go#L140

a black tunnel pixel and a black background pixel are indistinguishable, and any sprite drawn over the field temporarily corrupts the answer

I'm going to cheat here. If next levels of Digger have black pixels in the field background they are going to be almost black, the difference invisible to the eye. But the first level looks good.

Sprites are drawn in layers: https://github.com/geniot/digger/blob/main/src/scene_game.go#L61 , and so they do not corrupt anything.

And that is another big difference from the original game - how stuff is drawn. I wonder when a falling bag hits the digger and some monsters along the way how it all is going to look like? In the original game they all start to kind of blink overlapping each other pulled down together. But we'll see what I come up with when it comes to implementing this. Again: I'm not trying to make it 100% faithful, really hoping to make something better, smoother, AI smarter, etc. Playing in the browser, on desktop and a few of my handheld game consoles, such as Trimui: https://github.com/geniot/tsp-rubik

I considered creating a bidemensional array for tracking the field changes, I considered arrays for cells, but then I was constantly hitting the same question: if an image is an array itself why would I create another array that duplicates it? I am still not sure that Raylib's image is a good answer to the field collision detection but we'll see.

Monster AI is something that I'm going to look at more closely I guess. When it comes to it.

Thanks again for your answer! It really motivates me to work on this remaster further.

[kebl3541]

Your pushback is fair, and on the main point you were already ahead of my answer. You are not reading back from the GPU in the hot path: field.image is a CPU side Image kept in step with the render texture by the rl.ImageDraw next to the rl.DrawTexturePro in Field.draw, and the only LoadImageFromTexture is in Debug(), which is commented out at the call site. That was the part that mattered, and it is done.

You are also right to reject the cell grid. I pitched CellState[rows][cols] on the assumption you were reproducing the original's grid movement, and you are explicitly not: you want pixel perfect collision and smoother movement than the 1983 version. A coarse grid would take away the thing you are building. So drop that part of my answer.

But I would put the remaining question differently. "If an image is an array itself why would I create another array that duplicates it?" The array is not what gets duplicated. The rasterizer is. You now have two independent pieces of code that turn the same intent into pixels, DrawTexturePro on the GPU and ImageDraw on the CPU, and they have to agree forever, across driver versions and across your desktop and WASM builds. Debug() is the receipt for that: you wrote a pixel exact comparator with a panic in it because nothing else guarantees the two stay in sync.

One thing you raised that turns out not to be a problem: I checked cdblob.png, and it is pure black with binary alpha, only 0 and 255, no anti aliased edges. So the blend cannot produce almost black fringes at tunnel borders, and your plan to make later level backgrounds almost black instead of black is safe as far as the blob sprites are concerned. I was wrong to imply otherwise.

There is a real bug in IsColliding, though, and it is live rather than theoretical.

GetImageColor bounds checks and returns a zeroed Color when the coordinates fall outside the image. From rtextures.c:

Color GetImageColor(Image image, int x, int y)
{
    Color color = { 0 };

    if ((x >=0) && (x < image.width) && (y >= 0) && (y < image.height))
    {
        ...
    }
    else TRACELOG(LOG_WARNING, "IMAGE: Failed to get pixel data, x/y out of bounds");

    return color;
}

Your predicate is color.R != 0 || color.G != 0 || color.B != 0. Out of bounds gives {0,0,0,0}, which is indistinguishable from a dug tunnel, so anything past the edge of the 320x186 field reads as free space. Fire.Update calls IsColliding on both the spawn rect and the per step rect, so a shot travelling down a tunnel that reaches the field boundary never transitions to FireExploding; it keeps incrementing posX/posY off the field. Given the level plan puts the rightmost column at xp = 14*20 + 12 = 292 and the right blob adds another 16 plus a 20 wide sprite, tunnels do reach past x=320, so this is reachable rather than hypothetical. Worth guarding explicitly, treating outside the field as solid:

if x < 0 || y < 0 || x >= FieldWidth || y >= FieldHeight {
    return true
}

That is also the general shape of the argument for a plain Go array. Not a coarser one, the same pixel resolution you already use:

dug [FieldWidth][FieldHeight]bool   // exactly like MoveGrid.dots

You already have precisely this structure in MoveGrid.dots, at 320x186, and it costs about 60 KB. Three things follow from it. Go bounds checks it, so the failure above becomes a panic during testing rather than a monster silently walking through a wall. Indexing it is a plain memory read, whereas rl.GetImageColor in raylib-go is one cgo crossing per pixel that passes the whole Image struct by value, and IsColliding makes one call per pixel of the rect. And because the blobs have binary alpha and your destination coordinates are integers, you fill it by stamping a precomputed mask at an offset, which is not a rasterizer and cannot drift from the GPU the way ImageDraw can.

The last one is the reason I would still move off colours eventually, even though your current setup works. Black versus not black is one bit, and the field is about to need more than one. Once emeralds and money bags live in the field, "is this pixel passable" and "is this pixel an emerald" and "is this pixel earth that a bag can fall through" are different questions, and a uint8 per pixel answers all of them while a colour test answers none of them without you inventing a colour encoding and decoding it back out.

[geniot]

I totally agree with your arguments about memory, collisions, efficiency of using GetImageColor, etc.

But my problem was that I was too lazy to write my own eating algorithm. Thank you for writing this convincing post. I have just figured out that it's not difficult at all. And it brings all the benefits that you mentioned.

As I see it: I need to generate two dimensional masks (arrays) for each image blob that eats the field and then apply these masks at the right x,y to the field array.

[kebl3541]

That is exactly it, and the laziness instinct was sound. You should not write an eating algorithm. You should write a stamp, which is about fifteen lines.

You already keep image next to texture in TextureImage, and you already apply the rotate and the flips before LoadTextureFromImage, so the mask can be built in NewTextureImage from the same pixels the GPU gets:

type TextureImage struct {
    image   *rl.Image
    texture rl.Texture2D
    mask    [][]bool      // new
    width   float32
    height  float32
}

// after the rotate/flip block, before or after LoadTextureFromImage
cols := rl.LoadImageColors(textureImage.image)
w, h := int(textureImage.image.Width), int(textureImage.image.Height)
textureImage.mask = make([][]bool, w)
for x := range textureImage.mask {
    textureImage.mask[x] = make([]bool, h)
}
for y := 0; y < h; y++ {
    for x := 0; x < w; x++ {
        textureImage.mask[x][y] = cols[y*w+x].A > 127
    }
}
rl.UnloadImageColors(cols)

A > 127 rather than A == 255 only because it costs nothing and stops the mask silently emptying if you ever run a blob through ImageResize or add a sprite with soft edges. For the four blobs as they ship today the alpha is strictly 0 or 255, so the threshold never actually fires.

Then the stamp, which takes the source rect so it keeps working for the half blob cases in drawExt:

func (field *Field) carve(mask [][]bool, src rl.Rectangle, dstX, dstY int32) {
    sx, sy := int32(src.X), int32(src.Y)
    for j := int32(0); j < int32(src.Height); j++ {
        fy := dstY + j
        if fy < 0 || fy >= FieldHeight {
            continue
        }
        for i := int32(0); i < int32(src.Width); i++ {
            fx := dstX + i
            if fx < 0 || fx >= FieldWidth {
                continue
            }
            if mask[sx+i][sy+j] {
                field.dug[fx][fy] = true
            }
        }
    }
}

and it drops in exactly where the ImageDraw currently sits in Field.draw, so there is still one place where digging happens:

rl.DrawTexturePro(textureImage.texture, sourceRect, destRect, ZERO_VECTOR2, 0, rl.White)
field.carve(textureImage.mask, sourceRect, int32(destRect.X), int32(destRect.Y))

Three things worth knowing before you run it.

The clipping in carve is the same bug fix as the one in IsColliding, so do both. Off field has to read as solid, not as empty:

func (field *Field) IsColliding(rec rl.Rectangle) bool {
    x0, y0 := int32(rec.X), int32(rec.Y)
    for y := y0; y < y0+int32(rec.Height); y++ {
        for x := x0; x < x0+int32(rec.Width); x++ {
            if x < 0 || y < 0 || x >= FieldWidth || y >= FieldHeight {
                return true
            }
            if !field.dug[x][y] {
                return true
            }
        }
    }
    return false
}

No flip. dug is in the same top down space as field.image and as your posX/posY, and the negative height in textureSourceRec is purely about how the render texture gets blitted to the screen. It does not touch the array. This trips people up because the flip is visible in Debug(), where you have to ImageFlipVertical the clone before comparing.

Your source and destination rects are the same size at every call site today, so the stamp never has to scale. If you ever draw a blob at a different size than its source rect, that is the one place you would need to care.

One correction to something I implied earlier, since I went and looked: cback1.png is only two colours, (170,0,0) and (170,85,0), with no pure black in it at all. So your current R != 0 || G != 0 || B != 0 test is not actually fragile today, and the almost black plan for later levels would have worked. The case for dug is not that the colour test is broken, it is the bounds bug above plus the fact that one bit per pixel stops being enough the moment emeralds and bags go into the field. Worth being clear that you were not sitting on a live defect there.

If this answers the original question about RenderTexture and collision, marking it as the answer would help the next person who searches for it, since this thread is one of the few places the readback question gets a concrete alternative rather than just "don't do that".