So you want to create a UI System

So I had to do a research project last year I decided to do a UI system for Azul UI Depaul’s internal framework. Learned a lot about doing 2d on top of 3d and the different libraries available. Below my notes in case anyone wants to compare. At the end there are two documents attached that show usage and general design.

Introduction

The purpose of this document is to outline the process that got us to the result that we have today with PaletaUI we will highlight the problems and the solutions that were picked. We will also conclude on lessons that we go thru out the process. This differs from the Design Document which talks about what actually got constructed and from the usage guidelines which are meant for the end user if we have one for PaletaUI.

Topics

How to change azul to ortographic mode.

This was the first problem that we faced. In Azul or better said my custom version of azul we were not using the traditional camera but we had a system that involved our custom camera and a camera manager. We wanted to keep that functionality and still be able to render In 2d so what I decided was that in the rendering loop we were going to split the two parts so this problem became simpler to just how we set the camera. I implemented a method for orthogonal projection to pass to the shader manager in order to be able to handle this. The method is static in camera manager is called GetOrthogonal.

How to draw on top of a 3d scene

Like we mentioned above we had to split the rendering cycle into 2d and 3d. I wanted to isolate Paleta as much as possible from Azul. So I created a class called UIManager to be the external facing point for the drawing call. Then in render scene I call ui manager update and draw after the scene from Azul does its call. We decided that the 2d equivalent to the Scene was the View and that we were going to Have only one active at a time. Additionally the Scene was going to assign the view using UIManager and be responsible of filling it.

How to draw a quad

We have our call set and know we need to be able to make a square because the first object we were going to create was basically a panel. We used the same rendering technique used in the rest of Azul create a predefined Vao. The only difference is that we are not using a translation function but recreating the quad if it ever moves. This is due to the fact that on the 2d ui things hardly ever move.

How to Control Rendering Order

Basically drawing on the screen is a stack so we needed a way to order ourselves. The decision that was made was to add the frame basically and object that only purpose is to be read in the order it should be drawn so a view has many frames.

Different types of Settings needed for the quads

To be able to display Text , Backgrounds and images I had to use control flags to indicate when to load a texture and load the texture shader, the specific text character shader or simply draw solid. I handled inside the quad class.

How to load fonts

To load fonts we used the library freetype which allowed to grab fonts directly out of ttfs. All fonts load from the same folder the application is running from. Note in the visual studio solution is the main folder for the solution.

How to Store the Font

Decided to make a font manager that basically contains a hashmap of the loaded fonts one font basically means a font file in a specific size. We basically create a texture for every character.

When to Store the Font and for how long

We load them until someone tells us to unload. They are fairly expensive to read and turn into textures in memory so we are super lazy on letting them go this is handled on the FontManager Class

How to calculate position for the letters

This was pretty tricky we used a technique extremely similar to the one learnopengl.com We basically create a character struct to give us width height and the bearing so we can do proper spacing between characters. The logic to do this is inside label since is the class that everyone else that has text inherits.

What Shaders to use with Freetype

We had to create our own fragment and vertex shader. Basically what our shader does is that it looks at the image and uses the color red from the texture to set the alpha. The textures from freetype have Red characters with black backgrounds. Additionally it substitute any color that you send into it to replace the red so we can print in different colors.

What Structure should we use for the UI objects

Once we had figured out how to render text we were ready to actually start working on creating a compelling structure to place things on the screen. Added to the View and Frame decisions that had been previously made. I decided on the widget to be the base class for all representing a rectangular area of the screen and made the coordinate system to work from the bottom left. So we have two functions to basically push from the bottom and push left a widget.

How do we keep our structure open

It was my goal to allow people to create their own components later and to be able to make aggregated components like the highscore one in the demo. Due to this the widget class is an abstract that allows to be inherit and can redefine its draw and update so that it can be manipulated.

Is Panel a special type of widget

Not really but yes the difference with panel is that it works like an anchor basically allows you to add many widgets to it and anchor them to its x and y.

Are Buttons and Textbox special kinds of labels

It made sense for buttons to inherit from labels because they needed to show text and a background it ended up working very well. I also thought that textbox was a button and then realize that it was not since it only shared one characteristic that it could be clicked.

How do we take mouse input without getting too attached

To solve this problem I created a Virtual Mouse that needs to be updated every cycle. It works well and reduces all the data to normal ints.

How do we take a click

I did not made an elaborated collision system. Realized that you only have 5 o6 buttons or textboxes per screen decided to let them manage it themselves by doing a selfcheck on update. This is a possible area for improvement later on.

How do we take the actions that a button needs

Created a command system for this you need to inherit the class and create a specific command for your action. It ended up being great it encapsulates the action and makes them reusable. There are two commands implemented natively in the system on debug prints that something got clicked. The second one set focus to the widget that uses it.

How do we load the images for our system

I wanted to use Soil and add an image loader similar to the font manager but due to problems with SOIL ended up using the tga loader and texture manager from azul. So for now they need to be added to the enum. This is my number one area for improvement.

How do we take keyboard input without getting too attached

The same solution that we used for mouse would not work here. What we ended with was creating a keystroke observer that only has one subject which is whatever widget that is focused at the moment. Which created the problem of having to track and reset focus.

How do we keep track of Focus

We use focus to know where to send our keystrokes right now for the textboxes. There is a FocusManager class that gets set via the SetFocus Command Whenever a textbox is click other classes can use this command to set focus to themselves if they need it. The focus is no hierarchichal only one widget at a time.

What format should we use for declarations

I started on the route of using protocol buffers and thought it was a good idea had to do a reversal and fallback to xml for the following reasons. The compiler adding it to the solution was hard and while I solved this and got it running in the solution was worried that my setup would fail later. Second Reason and I think is the most important is that custom parsing of hierarchichal objects was really complicated. Seems like protocol buffers was meant to handle flat objects which makes sense for messages instead of nested hierarchy’s. With that said the format is pretty need and I think we should try to use it to create a duel networking system with protocol buffer serializing the messages. Is one of the projects I’m suggesting for the next semester.

I decided to use tinixml2 class to do deserialization.

How do we load the declarative format

Added a Load method on the View class and the frame class and that solved it for the top level objects. I only had to give a filepath to the view to be able to load an entire structure. With that said this created the problem that we have to be able to find the objects and ended up having to make the hasname interface that lets me search thru the frames and widgets.

How do we deserialize nested objects

This was an interesting problem and it was around the widgets ended implementing a singleton class widgetserializer which host a list widgetserializercomponents and uses a chain of command pattern to figure out who it matches with and executes it for that specific section.

How do we keep our serialization logic apart from our objects

We did not include the logic at all in the widgets but made a parallel structure in charge of loading them up.

How do we extend the serialization for custom components.

Widgetserializer has and add method to let you register widgetserializercomponents that were not created has part of the framework.

Lessons Learned

Libraries can take a while to integrate

If you abstract well you can forget about the underlying presentation

Let the user control lifetime of loaded objects

Deserialization != Construction

A few Ui Pieces can make an app a lot better looking

Abstractions for input can be complicated

Using Commands for Events is a good way of encapsulating logic

The Ui View is a separate element controlled by the scene

Improvements

Better image loading

Clipping to size.

Add Border to Widget

Add Gradients to background Circular and linear.

Parse Colors defined by rgba in the declaration xml.

References

1. http://voxels.blogspot.com/2015/07/opengl-game-gui-widgets-with-source.html

2. http://openglgui.sourceforge.net/

3. http://www.gamedev.net/topic/610816-how-to-make-my-own-gui-using-opengl-and-sdl-and-c-or-whateva/

4. http://learnopengl.com/#!Getting-started/Hello-Triangle

5. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=17&cad=rja&uact=8&ved=0CEkQFjAGOApqFQoTCL_r4b6up8cCFUZXPgodO60Ctg&url=http%3A%2F%2Fwww.uwplatt.edu%2Ffiles%2Fcsse%2Fcourses%2Fprev%2Fcsse411-materials%2Ff13%2Ffeldmeiera-gui%2520_programming.doc&ei=djzNVb_YI8au-QG72oqwCw&usg=AFQjCNEj0gnQbaiDdux75ToDv_RUtmfiog&sig2=uj47NywhsjTEZNaakMQjNQ&bvm=bv.99804247,d.cWw

6. http://www.codeproject.com/Articles/30775/TetroGL-An-OpenGL-Game-Tutorial-in-Cplusplus-for-W

7. https://developers.google.com/protocol-buffers/?hl=en

8. http://www.grinninglizard.com/tinyxml2docs/

9. OPEN GL SUPER BIBLE

Videos

Milestone 1- https://www.youtube.com/watch?v=9jAB_ZPbuyc

Final – https://www.youtube.com/watch?v=m42b73GEnrw&feature=youtu.be

Link to Design Document

https://www.scribd.com/doc/296137070/PaleTa-UIi-Design-Documentation

Link to usage guide

https://www.scribd.com/doc/296137071/PaleTa-ui-Documentation