Implementing Lua Scripting and Visual Scripting for Game Engine (Part 2)

At last, I finished the second part of this post. This is the second part of “Implementing Lua Scripting and Visual Scripting for Game Engine”. Check part 1.

This part will cover the hardest part and will be hard to explain. If you don’t get the first idea when you read it (me too! Damn! I’m bad at writing), bear with me and try to read again.

Visual Scripting

Struct Editor + Visual Scripting

The Visual Script editor is created on top of Maya using Qt and Python, so there is no C++ code in here. Even we created this on top of Maya, the editor doesn’t really depend on Maya library. So in other words, we can make the editor stand-alone. As I mentioned before, we have 2 types of Visual Script editor, one is for a struct and another one is for an entire scene. Once again, the difference is the scope of the scripting.

For implementing visual script components, we used base Qt Classes for drawing. the Qt has several classes that can be used, such as QGraphicsScene, QGraphicsItem, QGraphicsPathItem, and QGraphicsProxyWidget. QGraphicsScene is used as a scene, or you can think this as a canvas. QGraphicsItem is a base class for a thing that is drawn in a canvas/scene. QGraphicsPathItem is a specialized class from Qt for drawing a line. And last, QGraphicsProxyWidget is a proxy class for drawing basic form widget, such as text field, toggle, into a canvas/scene.

All those classes, we used to build new classes for our visual scripting purpose. These are classes that we created and the functionality of each class.

  • NodeScene, the class acts as a canvas to draw all nodes and lines. The class is inherited from QGraphicsScene.
  • NodeItem. This is a node class for creating a node in a scene. This class contains some basic node information like name of a node, type of connection and also list of parameter/attribute. This class is inherited from QGraphicsItem.
  • AttributeNodeItem. The node has its parameter, either input or output or both. This parameter represents AttributeNodeItem, which is drawn under NodeItem. This class is inherited from also from QGraphicsItem.
  • NodeSocket. An instance of this class handle connection between two nodes or two attributes. This class is also inherited from QGraphicsItem.
  • NodeLine. The name is pretty obvious, this will connect two NodeSockets. This class is inherited from QGraphicsPathItem.
  • GraphicsEditLine and GraphicsCheckbox. Both classes are used to draw the text input and checkbox — which is usually part of Qt form — in NodeScene or QGraphicsScene in general.

We created functions to parse to and from JSON file for all the custom classes. This makes loading and saving are really easy to implement. Other than that, we use the functions to also make Node can be defined or made by a single JSON file. For instance, take some arithmetic operation, such as float multiplication. I can define that as JSON format like below.

{
   "Name" : "a * b",
   "Conn" : "none",
   "Func" : "({1}*{2})",
   "Attrs" : [
      {
         "Name" : "res",
         "Type" : "float",
         "Conn" : "out"
      },
      {
         "Name" : "a",
         "Type" : "float",
         "Conn" : "in",
         "Constant" : true
      },
      {
         "Name" : "b",
         "Type" : "float",
         "Conn" : "in",
         "Constant" : true
      }
   ]
}

In the editor, it’ll look like this.

It is a very simple node. By seeing it’s visual, it can be easy to see the relation between JSON attributes and its visual. While “Func” in a JSON file is used to represent the node when converting to Lua script that will be explained later. If you notice there are some expressions in “Func” ( “{1}” and “{2}” ), the expression means the value for attribute’s value by index.

Also with the JSON functions, we created for the editor, we did create a bunch of node templates, from simple math to object movement and Physics support. Take an example of Physics raycast node in JSON format and its visual.

{
	"Name" : "RaycastLine",
	"Conn" : "both",
	"Func" : "Physics.RaycastLine(l_getGameContext(), {0}, {1}, {2}, {3})",
	"Attrs" : [
		{
			"Name" : "from",
			"Type" : "Object",
			"Conn" : "in"
		},
		{
			"Name" : "origin",
			"Type" : "vector",
			"Conn" : "in"
		},
		{
			"Name" : "direction",
			"Type" : "vector",
			"Conn" : "in"
		},
		{
			"Name" : "length",
			"Type" : "float",
			"Conn" : "in",
			"Constant" : true
		}
		
	]
}

Lua Script Integration

In this section, I’ll explain how to generate Lua script from the visual script editor, how the Lua script running in the C++ engine, and some tricks I used to handle access object between C++ game engine and Lua script.

First, let me tell you why don’t we use visual script JSON file and parse it into C++ game engine and let the engine handle the visual script without converting to Lua extension.  The first reason is because the engine we used is already integrated with Lua extension, so it’s not good if we don’t leverage its existence. The second reason is by building the script engine for the visual script is not doable for one-third of semester plus we have another class to worry (Damn, student excuse!). It also would give us headache and become sleepless zombie if we forced to do that. The last reason is more about file size and memory when load the JSON file itself. Obviously Lua script generated is smaller than JSON representation of whole visual script. Also I believe we can make Lua script as binary file and load that into the game engine with help from Lua-C extension. All those reasons are my excuses. On other hand, by implementing its own visual script engine itself makes it easy to us to implement more advanced features, like debugging the visual script.

To implement a generator for the visual script is relatively easy. We just need to traverse all node from the root node to all node by following the flow line (thick white line). There is an exception to our generator, it can’t handle cycle in the flow, so basically it should be a DAG (directed acyclic graph). Each time it stops on a node, it will grab the “Func” parameter and process that based on the parameter and put that into the generated Lua script. An example of the generated Lua script is below. It’s generated from the image of the visual script before.

function Update(struct)
	if (struct.isShooting) then
		if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 16)) then
		else
			Animation.SetAnimation(l_getGameContext(), struct.id, 16, true)
		end
	else
		if (Vector.LengthSquared(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))) > 0.01) then
			SceneNode.SetPosition(l_getGameContext(), struct.id, Vector.Add(SceneNode.GetPosition(l_getGameContext(), struct.id), Vector.MultScalar(Vector.Normalize(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))), (g_delta_time*struct.Speed))))
			SceneNode.TurnInDir(l_getGameContext(), struct.id, Vector.Normalize(Vector.Sub({ x = struct.target_x, y = struct.target_y, z = struct.target_z }, SceneNode.GetPosition(l_getGameContext(), struct.id))))
			if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 18)) then
			else
				Animation.SetAnimation(l_getGameContext(), struct.id, 18, true)
			end
		else
			if (Animation.IsAnimationPlaying(l_getGameContext(), struct.id, 17)) then
			else
				Animation.SetAnimation(l_getGameContext(), struct.id, 17, true)
			end
		end
	end
end

Implementing Lua into a game engine — regardless that Lua script already embedded to the engine — is not that hard. What we need a Lua library or API that works with C++. Hopefully, there is a good (I think it’s an official one) API that already implemented (see more on https://www.lua.org/pil/24.html). I won’t talk about how to implement the Lua script API and how that works in C or C++ code. But I will talk about some tricks I used to make it work with our visual script and our struct.

Unfortunately, the API doesn’t a reference to a parameter from C++ into Lua, it passes a value of a parameter. Also, it only handles primitive types, like string, number, int, and boolean so we can’t actually pass just a defined struct from C++ to Lua.  We need to create a table in Lua and put each member of a struct into the table. For our struct, besides generating the declaration of struct into a header file (see part 1), we also generate some helpful function to push the struct into Lua. This one is an example for our Soldier.

void Soldier::pushLuaStruct(lua_State* l, Soldier& s) 
{
	lua_newtable(l);

	PE_PUSH_STRING_TO_LUA(l, "StructName", Soldier::getName())
	PE_PUSH_STRING_TO_LUA(l, "Name", s.m_Name)
	PE_PUSH_NUMBER_TO_LUA(l, "Speed", s.m_Speed)
	PE_PUSH_NUMBER_TO_LUA(l, "Health", s.m_Health)
	PE_PUSH_STRING_TO_LUA(l, "Weapon", s.m_Weapon)
	PE_PUSH_NUMBER_TO_LUA(l, "Mana", s.m_Mana)
	PE_PUSH_PEUUID_TO_LUA(l, "Enemy", s.m_Enemy)
	PE_PUSH_NUMBER_TO_LUA(l, "Happiness", s.m_Happiness)
	PE_PUSH_BOOL_TO_LUA(l, "isShooting", s.m_isShooting)
	PE_PUSH_NUMBER_TO_LUA(l, "target_x", s.m_target_x)
	PE_PUSH_NUMBER_TO_LUA(l, "target_y", s.m_target_y)
	PE_PUSH_NUMBER_TO_LUA(l, "target_z", s.m_target_z)
}

PE_PUSH_<TYPE>_TO_LUA is a macro I created to simplify generating C++ file.

Then, we pass the struct into a Lua script. Then script start changing the struct. But like I mentioned, it only passes the value, it doesn’t matter how the script changing it, the engine still doesn’t know what is changed. We need the struct back after the script deals with it. To do that it’s needed to create a wrapper inside Lua script for a function and return the struct back after changing it.

function Update_Wrapper(struct)
	if(Update ~= nil) then
		Update(struct)
	end
	return struct
end

That’ how the struct pushed and pulled between C++ and Lua. Unfortunately, the struct itself is not representing the real object in the game world. It is something that we can use to modify the real object though. Another problem is how to access and modify the real object’s attributes. There is no easy way passing an instance of a class in C++ to Lua and access all attributes of the class freely (You already knew the way I pass the struct to C++). Technically, there are some ways to do it, but I’ll mention two basic and simple ways to do it. The first one is by passing the memory address of the object, but we need to make some helper functions in Lua that help to get or push attribute from an instance of C++ class. The other way is by passing a unique id from C++, and still, it needs some helper functions to access real object’s attributes. Basically, the engine has table that will point a unique id to real object. The unique id can be anything, it can be a string or integer or four integers or any struct. It has no significant advantages or disadvantages between two ways. Personally, I think using unique id is safer than passing the object pointer, since it’s not exposing the real memory address, and if the engine have some memory management complex functions, it’s still guaranteed that the id is never changed.

As I mentioned, the struct is attached to the real object in the game, so we can put the object id as an attribute of the struct itself (“struct.id”). After exposing the unique id through the struct, we need to make some functions to help access its attribute. This is an example of a function to get the position of an object.

SceneNode.GetPosition(l_getGameContext(), struct.id)

Function “l_getGameContext()” is a global function to get the “context” or instance of the game. Underneath the function, the implementation in C++ would be like this.

int l_getSceneNodePosition(lua_State* luaVM)
{
	// Get the parameter of a function
	PE::GameContext *pContext = (PE::GameContext*) (lua_touserdata(luaVM, -2)); // get first parameter *GameContext
	PEUUID uid = LuaGlue::readPEUUID(luaVM, -1); // get second parameter "struct.id"
	lua_pop(luaVM, 2);

	PE::Components::SceneNode* pSceneNode = getSceneNodeFromUID(pContext, uid); // searching the object on the table

	Vector3 result;
	if (pSceneNode) {
		result = pSceneNode->m_base.getPos(); // read attribute from the object
	}

	pushVector(luaVM, result); // return a result to Lua

	return 1;
}

I know it’s a very tedious process if you have a lot of attributes of the object that you really want to expose to Lua. But this is a learning process, I don’t mind to get messy for this.

I think that’s it for now. It’s too much to handle (both to read and to write). I’ll make the last part to discuss some future improvements and also the conclusion of the current system we developed.

Implementing Lua Scripting and Visual Scripting for Game Engine (Part 1)

Scripting is part of a game engine that makes really easy to implement gameplay of a certain game. Many of game engine nowadays really depends on Scripting, for example, Unity3D with its C#, Javascript and Boo; and Unreal Engine 4 with its Blueprint system. Visual scripting (like UE4) is a way to script your gameplay using nice flow chart/diagram. Scripting generally used by a designer to implement and ordered logic without dealing with the engine, or programmer to help to implement their logic using their feature implementation inside the game engine.

This semester (Spring 2017), I and my teammate, Jonathan, have been implementing our little tools to deal with scripting in Maya for USC’s game engine, PrimeEngine. This is a class project for Game Engine Tools Development. For the first time, we have no experience using Maya (except Jonathan), implementing tools in Maya using Python (it bites hard) and implementing scripting. But in the end, we have some clues how to do all of those.

This post is made as my dev blog, so I can remember what I made and how I made things in school or personal project. And maybe this post will be useful for someone who has no idea how scripting implemented and/or want to implement one.

I must note that in the class, we have 3 same things but with different approach and implementation. So this is not a perfect scripting ever. But it’s good to have an idea for one.

Here are some videos that show our current result of our visual scripting.

The Game Engine

The game engine that we use here is minimalist features engine compare to Unity3D and Unreal Engine (or any engine out there). It has at least Rendering, Event/Messaging System, partially Networking and also partial Lua integration (for pass value from a file into PrimeEngine).

So, if we’re planning to have a game with physics, we need to jump into the engine code and create/integrate physics engine for the game. I believe no one wants to do that, except the one who is really one to learn how a game engine works and implemented (like me). In other words, it’s good to have PrimeEngine if you want to learn and focus and game engine. I know some Studios here have their own custom engine, so learning from PrimeEngine is very valuable here.

Big Picture of the System

Big Picture of the Visual Scripting system

The diagram shows the big picture of the system. Mainly, the system has three layers in it: Maya, Python Modules, and the last part, the Game Engine Core (which consists all C++ and all runtime-related content).

The first layer is where we built all tools on top of Maya. Why Maya? Since PrimeEngine doesn’t have its own level editor. Our professor built tools in Maya to create assets and levels, then export them to PrimeEngine. In this layer, we add more tools supporting Visual Scripting, like Struct Editor, Node Editor, and Level Node Editor. Struct Editor is an editor for creating a struct for a certain type of object. Node Editor is a visual scripting editor for creating logic for a struct, which means that the Node Editor is related to a certain struct. The last one is Level Node Editor. Level Node Editor is actually same as Node Editor, except it’s for creating logic for the entire level or scene instead of on certain object.

The second layer is all python modules that help to export or convert all generated files from Maya (in the first layer) into game-ready assets or files and for some files we can do the opposite way.  In this modules, we have various modules, like Lua Generation, Struct Generation, Struct Reader, CPP Generation, Blob Writer, and Exporter. Lua Generation will generate Lua script from both Node Editor and Level Node Editor. Struct Generation will generate JSON file which is as a representation of struct definition. Struct Reader is a module that read a JSON file generated by Struct Generation and uses that as for other module and for Maya, such as filling the attribute for an object in Maya scenes. CPP generation will generate C++ class for a struct from a struct definition file. Blob writer will write binary blob file of an object in Maya scene that has a struct in it. The last module is Exporter. The exporter is usually used to export all assets in Maya into the game, but in this focus, the exporter will export a script that determines which blob and struct the object used.

In the last layer, we have our engine core. In this layer, PrimeEngine will use all generated files by the second layer and used inside the runtime.

Struct Editor

You can think about “struct” in C++ definition that contains variables and functions. But here, we focus to create a struct with only variables. Struct editor is an editor built on top of Maya to create this kind of C++ struct. Struct itself will be saved as a JSON file, so it can read easily back and forth to and from Maya.

Struct Editor (Left) and Struct Definition (Right)

The editor is a very simple editor. We can add, delete and modify variables inside the struct. Variable can be a float, int, bool, string or an object ( basically, a reference to an object in Maya scene). Once created, the variables of the struct will show up in the properties window of the object where a struct is attached.

Attached struct variables shows in Object Properties

Struct is added to the Maya object by adding struct definition (struct name, variables, version, and package) into Maya attribute. Maya has the ability to add any attribute to Maya object/node (For more info Maya addAttr). Once we added that to Maya object, we can see the attributes under extra attributes. The attributes and its value also saved into Maya file, so we don’t need to worry about saving and loading attributes.

In our struct definition, we have “version” (you can see that in “Struct Definition” image before). This version is used to check if the struct in Maya object has a different object or not. If it has, it will update attributes, by adding, deleting and modifying attributes.

Once we set up everything in the scene, it comes to export all object and run the scene inside the game runtime. Before we go to exporting struct, it’s good to know what representation of struct inside C++ engine code. Basically, all struct in Maya will be used by Header and CPP generation module to create the C++ struct. See figure below. That is the one example of Knight struct we used before.

Generated Header for struct Knight

Now we have that on the Engine side, we can export struct in a Maya object, into a binary blob. We use binary blob because we can easily convert the binary into a struct. The format for the binary blob is <8 bytes id>. The first 8 bytes are used for ID to determine which struct that should be used for the binary blob.

Next, in part two, I will continue to explain how we create Visual Scripting and Integration with PrimeEngine.