Forging a game engine - part one: the why, and the design

A forum for longer arguments surrounding how/if/to what extent an engine needs to be developed in order to facilitate the NGD project. This stuff might get quite technical at times, and I would ask to restrain from giving opinionated remarks and concise yourselves to more factual arguments for an ultimate decision to be made
Post Reply
User avatar
David Buunk
Posts: 75
Joined: Thu Jun 22, 2017 4:46 pm
Location: Netherlands

Forging a game engine - part one: the why, and the design

Post by David Buunk » Sat Jun 24, 2017 7:31 pm

Most existing game engines focus on a different type of game. They simulate a player in a world, not a world with a player. In most games what you see is the simulation. In grand strategy, the simulation is some internal data, and the map a mere representation of the simulation. You can change the graphical output from a 2D flat map to a 3D globe, without the underlying game being any different. You cannot do that with most other games. Of course, you can take an existing engine and use it for something different than it was designed for. But this takes a lot of extra effort, and is that worth it? I talked with Demian about this, and we agreed that I would try to make an engine geared towards a SOTE-like game. If I succeed and Demian likes it, it gets used.

What does a game engine do? It is really the level below the "mechanics" level. It doesn't care about a "Beaver with top-hat and monocle"-race, it also doesn't care about the concept of "race", or the concept of "pop". It cares about generic game objects and the methods of them interacting. The game engine is about the process running though the simulation in a multi-threaded way, and about the general principles of savegames and storing game data. The other thing that comes with the game engine is making sure there is a proper mapper, so that the simulation data can be viewed in a graphical manner.

Now, on to the design choices.

The simulation is done in a series of steps. A step may be a day, as it is in most games made by Paradox. A step may be a turn of years, as it is in the Civilisation series. The engine doesn't really care about what the real-world equivalent of a step is. During a step, a part or all of the objects in the game world get to do stuff. (A forest tile may grow some more biomass, a population of beavers may work on building a dam, etc.) But such an object doesn't exist in a void, what it does depends on other objects around it, and its actions change the objects around it. This presents two problems. Firstly, objects will be dealt with in sequence, so once an object decides what to do, some things around it will be updated, others won't. This will mean that outcomes are now dependent on some arbitrary order of execution, and that just feels wrong. The second is a more serious issue and shows up due to multi-threading. What happens if two threads try to change one variable at the same time? They both read it, update it, and store it. One of the threads "wins" and gets to update the variable last, the other thread loses and its changes are ignored. This is very BAD! There exists solutions for this kind of thing in the form of "locks", but those have their own issues.

My solution for those problems is as follows. All simulation steps get split into a "tick" and a "tock" phase. In the "tick"-phase every object decides what is to be done, but doesn't change anything, except its own private variables that no other object can get the value of. In the "tock"-phase all decisions get executed. How do objects communicate what they want updated? Through messages that get sent during the "tick" phase, and get executed during some "tock" phase. By carefully grouping the objects and the messages, each object can only receive messages from a single thread, preventing the second problem.

Internally, objects will refer to each other using pointers, which are really just number that describe a location in memory. This creates a problem with saving/restoring. There is no realistic way to ensure every object ends up in the same memory location it used to be. Thus, there must be a way to identify all game objects that is persistent between save/reload. For all static data objects, like "terrain types", or "races", this is no problem, we just give them an ID string, and have a dictionary to look them up in. Messages don't need IDs, since the only pointer to them is in the list of messages. Tiles are simple too, just use co-ordinates. But for all the other dynamic game objects we need an ID system. How to assign them, what format are they in. Have a single series of IDs, or multiple, one for each type of object? I'm still trying to figure out this one.

The final subject is what to do when dynamic game objects (like tile groups, or pops) die. All humans of a certain population may die out, a forest my disappear. You would need to make sure all pointers to it go away as well. For most purposes objects know what refers to them and vice-versa. A population would know in what province it lives, and the province would know all the pops living it it. Those pointers should be removed/invalidated by the main game code, and can be told by the engine to check their pointers. That doesn't go for the messages. So what to do about it? The solution I came up with is allowing all dynamic world objects to set a replacement. So if a province gets merged into another province, the "dead" province would point to it's successor, and all messages get delivered to this replacement. This, would, however mean that "dead" objects remain in memory, clogging it as time goes on. Once in a while this would need to be cleaned up, all objects will be told to update their pointers, all pointers in messages would be set to the replacement, and the objects would be deleted. Possibly, object IDs would need to be re-set as well.
Programming SotE.

Dimitri
Posts: 1
Joined: Sun Jun 25, 2017 12:31 am

Re: Forging a game engine - part one: the why, and the design

Post by Dimitri » Sun Jun 25, 2017 1:00 am

This is a great idea, and something I know I've struggled with when wanting to build my own Grand Strategy game. I've been working in 'Duality', but its not ideal for a GSG.

This may be a little preemptive, but will this engine be available beyond Songs of the Eons, for people who want to use it for their own projects?

And also, will the engine be open source so people can contribute to it?

I ask because I've wanted a Grand Strategy engine for a while, but lack the technical aptitude to make one myself. I think this could be the first publicly available, true Grand Strategy engine.

User avatar
David Buunk
Posts: 75
Joined: Thu Jun 22, 2017 4:46 pm
Location: Netherlands

Re: Forging a game engine - part one: the why, and the design

Post by David Buunk » Sun Jun 25, 2017 10:53 pm

Dimitri wrote:
Sun Jun 25, 2017 1:00 am
This may be a little preemptive, but will this engine be available beyond Songs of the Eons, for people who want to use it for their own projects?

And also, will the engine be open source so people can contribute to it?
Well, those questions are linked, and the answer would depend on what happens next.

I'm going to try to make an engine. If what I come up with is good enough for both me and Demian, it will get used and it becomes part of the SOTE code-base. So the answer would then depend on what happens with SOTE itself, and Demian decides that. Personally, I would think the source (of all SOTE) should be released shortly after the release of the first version.

If what I come up with is good enough for me, but not for Demian, I might see if there is interest in releasing it as a stand-alone. If there is I would release it, probably under GPL.
Dimitri wrote:
Sun Jun 25, 2017 1:00 am
I ask because I've wanted a Grand Strategy engine for a while, but lack the technical aptitude to make one myself. I think this could be the first publicly available, true Grand Strategy engine.
Don't expect too much of it, as I plan to gear it towards the needs of SOTE, and won't add functionality that SOTE doesn't need. On the other hand, if you are stuck with the same problem as Demian - not knowing where to get started without and engine - it might very well help you.
Programming SotE.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest