By overlaying data from AMG8833 thermal sensor on top of the Adafruit Memento camera viewfinder, I've successfully turned it into a thermal camera. The bad news is all of my bitmap manipulation code runs very slowly, bogging the system down to roughly a single frame per second. I blame my habit or writing Python code as if I were writing C code. Running tight loops shuffling bits around is fine in C, but now the same approach is incurring a lot of Python runtime overhead.

As I understand Python, the correct approach is to utilize libraries to handle performance-critical operations. My Python code is supposed to convey what I want to happen at a higher level, and the library translates it into low-level native code that runs far faster. In this context I believed I needed CircuitPython displayio sprite compositing engine to assemble my thermal overlay instead of doing it myself.

The viewfinder image is pretty straightforward, loading OV5640 into a Bitmap which went into a TileGrid as a single full-screen entry. The fun part is the thermal overlay. I created a TileGrid of 8x8 tiles, matching thermal sensor output data points. I then created another bitmap in code corresponding to my range of thermal colors. I didn't see any option for alpha blending in displayio and, as I believed it to be computationally expensive, I wanted to avoid doing that anyway. My palette bitmap is again a screen door of my thermal color alternating with the color marked as transparent so viewfinder image can show through.

In theory, this means every thermal sensor update only requires updating tile indices for my 8x8 TileGrid, and displayio will pull in the correct 30x30 pixel bitmap tile to use as a sprite rendering my 240x240 pixel thermal overlay. The underlying native code should execute this as native code memory operation far faster than my loop in Python setting bitmap pixels one by one.

I had high hopes, but I was hugely disappointed when it started running. My use of TileGrid did not make things faster, in fact it made things slower. What went wrong? My best hypothesis is that compositing tiles with transparent pixels incur more workload than I had assumed. I also considered whether I incurred color conversion overhead during compositing, but as documentation for displayio.Palette claimed: "Colors are transformed to the display’s format internally to save memory." So in theory color conversion should have been done once during startup when I created the thermal color tiles, not during the performance-critical loop.

The upside of Python's "offload details to libraries" approach is that I don't have to understand a library's internals to gain its benefits. But the corresponding downside is that when things go wrong, I can't figure out why. I have no idea how to get insight into displayio internals to see what part of the pipeline is taking far longer than I expected. Perhaps I will eventually gain an intuition of what is quick versus what is computationally expensive to do in displayio, but today it is a bust and I have to try something else.

https://github.com/Roger-random/circuitpython_tests/commit/650f46e64bc08de9f8c1f451a4d18ea7021e92fb