BearLibTerminal: a pseudo-terminal window library for roguelike

[Omnivore]

2. There is one very internal thing. I've tried to make the library at least halfway thread-safe so as long as you do not call functions concurrently, which thread they are called from does not matter. This was achieved by running input and rendering from a separate hidden thread. However, on some platforms (e. g. OS X) you just cannot work with windows (i. e. input) from non-main thread and on some others (e .g. X11) things are not guaranteed to work reliably, though they usually does. This will not require to alter the API but some functions would be restricted to be run from the main thread (open/close, set, has_input, peek, read, read_str). You'll still be able to invoke all other functions (including refresh) from whatever thread. I just wonder how many people even thought about multi-threaded interface processing in terminal-like application >_< it's possible no one will even notice.

Ah, this explains the delayed shutdown.  I'd been wondering why the process sticks around for half a minute after the window closes.

Frankly, it's over-engineering, in my opinion.   Most UI frameworks I've worked with in the past have required access through only the main thread for a variety of reasons.  I suspect that anyone considering the use of multi-threading would expect that and be aware of the workarounds. 

For a roguelike, I believe multithreading is overkill.  I've run tests on some of the heaviest algorithms in use for roguelikes and they really don't benefit from parallelization (via multithreading or otherwise) at the present time.  Context switch cost along with data exchange cost just destroys any performance gain.  If someone is multithreading for other reasons, most languages today have some form of lightweight thread-like mechanism available that mostly alleviates the need.

The only case that immediately springs to mind where multithreading might conceivably be worth the headache would be in procedural generation of maps.  Even then, it is quite possible to use lightweight mechanisms to emulate the desired behavior.  In any case, I can't see any reason why such a background task would need to communicate with the UI.

As the adage goes, if you think you have a problem that will be solved by multiple threads, you now have two problems.

Just my 2c worth,
Brian aka Omnivore

[Cfyz]

Ah, this explains the delayed shutdown.  I'd been wondering why the process sticks around for half a minute after the window closes.

Wait, what? The window with all its resources (thread included) should be completely released after terminal_close. Do you have some scenario reproducing this behavior?

Frankly, it's over-engineering, in my opinion.

Indeed it is, I just thought I would be able to cover a few more cases at some reasonable cost and it turned out the cost is big. As for parallelization, what I meant was more in the direction of rendering offload, when world processing (model) is done separately from drawing its state (view). There is even an example of it in this very thread (notostraca's test a few pages ago). It is fairly common approach in more complex games or visualization applications, not sure to which extent it applies to roguelikes. Probably some sandbox-style game with lots of actors (with their own task queues) would benefit from this.

Most of the tasks I can come up with to run in parallel do not interact with rendering/interface directly. As you mentioned, it's hard to parallelize efficiently usual algorithms so I'd offload something more high-level like chunked world generation for seamless travel or continuous processing of world objects while they are out of sight.

[Omnivore]

Ah, this explains the delayed shutdown.  I'd been wondering why the process sticks around for half a minute after the window closes.

Wait, what? The window with all its resources (thread included) should be completely released after terminal_close. Do you have some scenario reproducing this behavior?

Windows 8.1, 64bit OS, 32bit Python 2.7.9, BearLibTerminal_0121 32bit dll, simple script just loading fonts, displaying a screen, and the normal event loop that falls through to close() after receiving TK_CLOSE event from read().  Script ends, nothing special after calling close().  The terminal window closes promptly, but process hangs around for around 30 seconds before returning to the command prompt.

Hope that helps,
Brian aka Omnivore

[LisacPisac]

I have a small, but I believe important request. When setting the window size, IE. to 120x60, could you make it so that it doesn't create a [60][120] arrangement, but a [120][60] one, since it's more natural because of arrays. That is to say, you don't address column, then row, but rather row and then column as is the case with arrays.

Thanks!

[Cfyz]

LisacPisac, sorry but I don't understand what you're talking about. With simple put(x, y, c) and print(x, y, s) functions you can't even see if there is any underlying array, let alone how it is allocated. You can fill cells in any order you want.

[LisacPisac]

Right, but what if I am using a standard array to store data about the map? The array uses a [row][column] format, where as the terminal itself uses a [column][row] format. And so, if I am to translate data from the array into the terminal in the right order, using two for loops using counters 'i' and 'j', I address my array as array[j]. Which is counter intuitive.

Okay, granted, perhaps I am doing something TERRIBLY WRONG and don't realise

[Omnivore]

There's no fundamental reason, for a rogue-like, to use row, column format storage arrays.  Your data accesses will tend to be rectangular - changing in both x and y so there's no real speed or cache difference between x, y and y, x formats.   If it is causing you problems, switch your storage array to use a [column][row] format.

More and more these days, I don't even use arrays for map data storage.  I suspect I'm not alone.  Hash maps, sets, and simple lists using immutable coordinate keys and object ids are a solid alternative.  Particularly with layered output, the 'map' never exists as a singular entity until it hits the screen. 

Hope this helps,
Brian aka Omnivore - and Happy Holidays to all

[Quendus]

OK, Terminal.PutExt is just awesome.

I wonder if anyone using corner-coloring in put_ext. I hope no one is. It does not work uniformly (with final result depending strongly on particular hardware, ugh) and restricts some possible optimizations, so I plan to remove it.

Is this still going to happen? I wouldn't miss the corner-colouring feature at all, because its main effect is to make me use an array of four copies of a single colour whenever I use put_ext. A single colour argument would be much preferable.

[Cfyz]

its main effect is to make me use an array of four copies of a single colour whenever I use put_ext. A single colour argument would be much preferable.

But you can pass NULL there to use the current foreground color...

[Quendus]

When drawing tile maps, I prefer to specify each tile's colour in the call to put_ext. It just feels neater (even with the weird requirement to make an array of colours). I would do the same with layer if it was an optional parameter in put_ext.

Leaving the foreground colour alone also means I don't have to worry about accidentally drawing strings in the wrong colour (as old versions of DoomRL, NPPAngband(?), et al did). Colour as a global persistent state variable just feels like an unnecessary legacy from ncurses.

[Cfyz]

When I said remove I meant for good. Output attributes as non-persistent parameters is understandable idea and helps performance (about 30% actually). But there are several attributes: foreground and background colors, layer index and replace/combine 'composition' flag. Adding everything to the function looks clunky, put_ext then will have nine arguments! But leaving one or two of them will look inconsistent. Why only foreground color? Why there is no such color argument in the regular put then? I don't see a clear solution for this.

Personally I'd just wrap it in the two-line function. This won't help with miscoloring but that itself is arguable as it's better to always set every relevant state before a logically separate action.

[Omnivore]

Not sure the complexity it worth it as it can just as easily be done in user code, but...  you could define a state structure and, to avoid headaches in passing structures back and forth- add a set of state management functions.  create_state, select_state as a minimum.  Then the user would create a state for a given type of operation and select that state during the op.  Could have select_state return the current 'old' state, then the user could just swap contexts.  Its similar to how GDI in Windows is managed in some frameworks.  Each state would hold everything except the character/symbol information.

From my perspective, I'm just as happy to roll my own user side state management - if I need it. 

Hope this helps,
Brian aka Omnivore

PS: Could go as far as: put_ext(x, y, char, state_token); the create_state call could copy the current state so the user would only change what differs in the new state.  Sounds simple, I'm probably overlooking something!

[Cfyz]

Here we go, version number thirteen, codenamed "better late than never". OS X is supported now.

As it was mentioned before, OS X support necessitated some changes so library is now single-threaded and must be used from the main thread only. I do have a solution to make drawing possible to be used from another thread (nope, not a global lock :-)), but it will introduce slight overhead in library operation and I am not sure if this feature even needed. Most of the time the code can be easily pseudo-parallelized anyway. For now, attempt to call some key functions from a non-main thread will result in window closing (mostly correctly though abruptly) and a relevant message in the log.

Windows and Linux versions do not change much (aside from threading) and OS X version archive includes a dylib with library_name=@executable_path and an awkward SampleOmni.app bundle.

Next is full unicode codespace and a sane additional font management.

[Quendus]

It's great that BLT games can be run on OSX now. Thanks for all your hard work.
I've found a few issues while testing it on an OSX virtualbox:
* PyBearLibTerminal.py hasn't been updated to find the .dylib file.
* Keyboard input pages in SampleOmni is quite strange - it seems to read several keys (arrows, shift, w, a?) as being held down. This could be due to problems in virtualbox, though.
* In the previous version, I remember mouse events being passed to read() by default. Either this default has changed or mouse input doesn't work (but this could be virtualbox)
* While experimenting in the Python repl, I was able to read the correct value from state(TK_SHIFT). However, state(TK_CONTROL) was always 0, whether I held down the control key or the windows/apple key. Again, it might be due to problems in virtualbox. However, it does raise the issue of how BLT should respond to the apple key. I see that the windows and alt keys are ignored by design - on OSX, it might make more sense to ignore the control and alt keys.

[Cfyz]

* PyBearLibTerminal.py hasn't been updated to find the .dylib file

Whoops. Fixed than for Python and Ruby wrappers.

I'm thinking about making a pip package for python, one that will include library binary as well. Did you know that not long ago they made it possible to include linux binaries in python wheels? What I'm not sure about is what package-module structure to use. Namespaced packages which would allow "from bearlib import terminal" (leaving a chance to add another BearLibSomething, particularly map generation) are cool but reported to be fragile (pip issue #3 -- does this matter? I do not use Python enough to be sure). The other choice is 'from bearlibterminal import terminal' which is just a bit clunky.

* In the previous version, I remember mouse events being passed to read() by default. Either this default has changed or mouse input doesn't work (but this could be virtualbox)

Yeah, I went and changed the defaults which was a bit hasty and wasn't even mentioned, sorry; reverted, reuploaded, 0.13.1. I do think input filter should have a default value, though. A lot of the time you need keypresses only and filtering everything out makes input logic cleaner in simple cases (e. g. sketching or beginners' programs).

* Keyboard input pages in SampleOmni is quite strange - it seems to read several keys (arrows, shift, w, a?) as being held down. This could be due to problems in virtualbox, though.
* While experimenting in the Python repl, I was able to read the correct value from state(TK_SHIFT). However, state(TK_CONTROL) was always 0, whether I held down the control key or the windows/apple key.

Please check the fixed version; at the very least, SampleOmni might've looked like some keys have been stuck because it wasn't getting keyup events. I do not see any problems with keyboard input in my VMWare virtual machine though (even reading the state of Ctrl from Python), will try to run on a real Mac.

However, it does raise the issue of how BLT should respond to the apple key. I see that the windows and alt keys are ignored by design - on OSX, it might make more sense to ignore the control and alt keys.

It does look like the library should provide Cmd key access as it is seems to be what Ctrl is to Windows. I do not think ignoring Ctrl is correct though, it will  make porting apps harder.

Also, Omnivore, are you here? I've meant my response on bitbucket as an invitation to discussion. Can you provide some examples of what are trying to do? Maybe there are even some conventions about keys and shortcuts in cross-platform applications on Mac?