Matt Tuttle


No Comments

Build Configuration Files for HXCPP

If you've ever attempted to write an extension for Haxe using hxcpp you probably hit a wall trying to figure out how to create a Build.xml file. The common "solution" is to copy and paste what other projects have done until something works. However, if you want to really understand the inner workings of hxcpp's xml format you'll want to continue reading.

The Most Basic Build.xml File

First, instead of diving in to all of the specifics of the file format let's look at the bare minimum example of a Build.xml file.

Build.xml

<xml> <!-- Theoretically this tag could be named anything but hxcpp uses "xml" so we'll use that too -->
    <!-- A target with the id "default" must be defined or hxcpp will issue a warning -->
    <target id="default"></target>
</xml>

If you created this file it can be "compiled" using the command haxelib run hxcpp Build.xml. If the hxcpp command can't be found make sure that you have hxcpp installed using haxelib install hxcpp. For now it won't actually do anything but hxcpp will gladly read this file and not print out any errors or warnings.

Before we cover targets, and how they can be used, let's take a look at some of the other root level elements that can be defined.

<echo value="Can anyone hear me?" />

Echo is only allowed at the base level of the xml file. It's useful for printing out detailed information about what is being compiled.

<error value="This message will self destruct in... NOW." />

You can cause an intentional error to occur in hxcpp with this element. Why would you want to do that? Perhaps you don't want to support certain build conditions so instead you decide to show a useful error message to the person compiling your project.

Define Variables

<set name="foo" value="bar" /> <!-- define "foo" with the value "bar" -->
<unset name="foo" /> <!-- remove the "foo" define -->

<setenv name="foo" value="bar" /> <!-- define "foo" again and set an environment variable -->

I decided to combine set and setenv because they are easy to get mixed up. First, set and unset are used to assign and remove defined values. These are useful for conditional checks as we will see in a moment. The difference between set and setenv is that the latter not only defines a value but also sets it in the operating system environment variables. Note that all of the attributes shown above are required and hxcpp will fail if you forget them.

<!-- NOTE: this uses the "name" attribute and not "value" like you may expect -->
<path name="/path/to/binaries" /> <!-- append to PATH variable -->

The path element works almost identically to setenv with one major difference. It appends to the PATH variable instead of replacing it.

Conditional Attributes

Every element in the build.xml file can have conditional attributes applied to it. These will help customize the build for each operating system you may be targeting. There are only a handful so let's take a look at each one.

If

<set name="on_boat" value="yep" />
<!-- only checks for the definition of a variable not the actual value -->
<echo value="I'm on a boat" if="on_boat" />
<!-- multiple conditions can be chained together with or statements "||" -->
<echo value="Apple fanboys forever!" if="macos || ios" />
<!-- separating values by spaces means that both must be defined to pass the condition check -->
<echo value="Compiling for a 64-bit linux machine" if="linux HXCPP_M64" />

You can add the if attribute to an element to check when values have been defined.

Unless

<set name="have_pants" value="false" />
<!-- this will not execute because the condition passes, note that the value doesn't matter -->
<echo value="I can't find my pants" unless="have_pants" />
<!-- multiple conditions can be chained together with or statements "||" -->
<echo value="We don't like Apple fanboys!" unless="macos || ios" />

unless is the "everything but" condition. Useful for when you want to exclude certain portions of your build configuration.

IfExists

<echo value="Phew, it exists." ifExists="path/to/file.txt" />

This is a special condition for checking if a file exists in the file system.

Grouping Configurations and Including Other Files

<section if="windows">
    <echo "I'm in another section!" />
</section>
<!-- You can identify sections with an id attribute. This is useful for including in other xml files -->
<section id="my-section"></section>

You can use the section element to define a group of elements. It can have conditions like all the other elements so it's a good way to skip large portions of the build file.

<include name="include/more.xml" />
<!-- Adding the noerror attribute will prevent an error if the include file doesn't exist -->
<include name="does_not_exist.xml" noerror="true" />
<!-- The section attribute lets you restrict the include to a specific section. You must have a coresponding id in the included file. -->
<include name="other.xml" section="my-section" />

Including other configuration files is a good way to split up your build process as it grows larger. You may also want to split out platform specific configuration details into another file as well.

Miscellaneous Top Level Elements

Before we get to defining targets I want to cover the rest of the other top level elements.

<copyFile name="README.md" from="." />
<!-- Normally copyFile will fail if a file doesn't exist. "allowMissing" prevents that error. -->
<copyFile name="graphics/logo.png" from="assets" allowMissing="true" />
<!-- Adding "toolId" only runs copy file on targets where the toolId attributes match -->
<copyFile name="logo.png" from="assets/graphics" toolId="dll" />

Contrary to what you may think this doesn't copy files immediately, it only happens when a target executes. The name attribute should be a filename and the from attribute is the directory where a file of that name exists. The file will be placed in the target's output directory with the same filename. If you include a directory in the name attribute it has the potential to fail because hxcpp does not create folders for you.

<!-- allowed name values (androidNdk, blackberry, msvc, mingw, emscripten) -->
<setup name="androidNdk" />

The setup element is used to set defines for specific build environments. It takes a name for a build environment and then passes the currently defined values to that tool.

<pleaseUpdateHxcppTool version="0.1" />

Used by hxcpp when the version is outdated. I would advise against using this element unless you know what you're doing.

Targets and File Groups

Targets are the bread and butter of hxcpp's configuration files. Think of a target as a group of executed commands to eventually either build an executable or some other form of output. Let's go back to the basic example from the beginning of this post for a second.

<xml>
    <target id="default"></target>
</xml>

So we can see that a target at the most basic sense only requires an id attribute. It's also worth noting that the "default" target is important because it specifies the starting point for hxcpp. Without a default target hxcpp will display an error message. Let's add a bit to this example.

Hello.cpp

#include <iostream>
int main(int argc, char *argv[])
{
    std::cout << "Hello world!" << std::endl;
    return 0;
}

Build.xml

<xml>
    <!-- This is a file group, it contains a set of source files -->
    <files id="common">
        <file name="Hello.cpp" />
    </files>
    <target id="default" output="hello" tool="linker" toolid="exe">
        <files id="common" /> <!-- reference the file group we just created -->
    </target>
</xml>

If you create a Hello.cpp file with the code above and run haxelib run hxcpp Build.xml you'll see that it generates an executable file. Run that executable and you will see Hello world! in the console window. You just compiled a C++ program using hxcpp!

Let's dissect this a bit. First there is a file group, which in this case is just our main C++ file. Following that is our default target but it has a bunch more attributes added to it. The output attribute determines what the final output will be named. If you are on Windows you may have noticed that it automatically adds a ".exe" extension. The tool and toolid attributes work together to determine how hxcpp will compile the code. The only supported value for tool is linker right now and the toolid can be a variety of values (exe, dll, static_link are the most common). You can make your own custom linker tool but I'm not going to cover that in this post.

Compiler Flags

If you've compiled anything beyond the most basic programs in C++ you've probably hit the need to include libraries in your program. Thankfully hxcpp supports external libraries as we'll see below.

<target id="opengl">
    <files id="opengl" />
    <!-- compiler flags -->
    <flag value="-I/usr/local/include" />
    <lib name="-lgl" unless="macos" />
    <vflag name="-framework" value="OpenGL" if="macos" />
</target>

These flags can get a bit confusing because they all basically do the same thing. The main difference between the flag elements and lib is that the former comes before the objects passed to the compiler and the latter is placed after the objects.

Also, flag and vflag are nearly identical except that vflag takes two values and merges them together with a space. It's important to note that you should not add spaces in these values unless you want them to be wrapped in quotes. Which also means that sometimes you have to use workarounds like add multiple lib elements for flags that require spaces.

Directories

<target id="directories">
    <outdir name="build" /> <!-- directory to place final output -->
    <builddir name="source" /> <!-- directory where source files are -->
    <dir name="obj" /> <!-- mentioned in the hxcpp source but never used... -->
</target>

As you can see there are several directory options that can be set. The first is the outdir element that specifies where to put the final output, from the linker step. It should be noted that the outdir name has a forward slash, "/", appended to it. The builddir can be thought of as the base directory. It is where you will find the files you want to compile and it will set the base for the outdir as well.

Dependencies

<target id="misc">
    <depend name="parent" /> <!-- requires another target to finish before this one -->
    <section if="linux"> <!-- works just like the root section -->
        <ext value=".so" /> <!-- add an extension to the output -->
    </section>
</target>

Like other parts of the build configuration, targets can have sections as well. This will group target configuration options just like you could group options at the root level. I've also added the depend element which tells the build tool that it needs to finish a different target before running the current one. Note that this does not use the id attribute but instead uses name.

Using With Haxe

Up until this point we've been using hxcpp to compile a C++ file directly. This is a perfectly viable way to build C++ and could be used in place of makefiles. However, you are probably wondering how this ties in with the Haxe language. So let's take a look at an example.

Main.hx

@:buildXml('<echo value="I added something to Build.xml!" />
<target id="haxe">
    <lib name="-lgl" />
</target>')
class Main
{
    public static function main() { }
}

build.hxml

-cpp out
-main Main

The buildXml metadata allows you to insert additional elements at the bottom of the generated Build.xml file. Take a look at out/Build.xml and scroll to the bottom of the file and you will see what was added. When you compile you should see the echo line show up in the output.

Now you may be wondering why there is a target with the id of "haxe". This is a special target id that Haxe creates when transpiling to C++. One thing that hasn't been mentioned so far is that targets can be appended to if they have the same id value.

Appending and Overriding Targets

If two targets have the same id value they will be appended by default. You can change this by setting an override attribute. See the example below for clarification on how these attributes work.

<xml>
    <target id="foo"></target>
    <target id="foo">
        <!-- append to foo -->
    </target>
    <target id="foo" overwrite="true">
        <!-- remove foo's contents and replace them -->
    </target>
    <target id="foo" append="true">
        <!-- append to foo (same as not having the attribute) -->
    </target>
</xml>

You may want to append or override targets when including other build configuration files. This gives you a lot of flexibility in how you compile your project.

What's Next?

This post covered a lot of the basics of hxcpp's xml file format. The information covered should get you started and hopefully provide a clear explanation on many of the elements found in Build.xml files. In future posts I'll cover how to use a Build.xml file to compile a Haxe extension and also the inner workings of hxcpp's linker and compiler elements.

Posted in Haxe

Tags: hxcpp


No Comments

HXML Overview for Haxe

If you've just started using Haxe or have been using OpenFL or Kha to build your projects you may not be familiar with Haxe's built-in build files, hxml. They use the same syntax as the arguments you'd pass to Haxe on the command line with a few other nice features built in. Even if you are a seasoned pro there may be one or two features you didn't know about in this post.

A Basic Example

Haxe can be overwhelming at first because of the number of targets it has available. Instead of trying to cover all of the targets this overview will just cover a few of them. Neko is a good target for command line tools and testing since it can be compiled quickly. To get started you should create a new project folder with the following two files in it.

Main.hx

class Main {
    public static function main() {
        trace("hello world");
    }
}

build.hxml

-neko output.n  # this is the compiled neko binary
-main Main      # your main class (must have static function main defined)

To build this example you enter haxe build.hxml on the command line. If everything goes as planned you should end up with a compiled neko binary named output.n in the same folder. To run this file you can enter neko output.n on the command line and it should print "hello world". If you aren't getting results you may want to make sure Haxe and Neko have been properly installed and that you typed everything as shown above.

Class Paths and Running Programs

So far you have a very simple project compiling in Haxe but what if you want to put Main.hx in a different folder? This is where class paths come in. Haxe needs a way to find the classes you create so it can compile them. You should create a source folder now and move Main.hx into it. Now, update the build.hxml file to look like the one below.

-neko output.n
-main Main
-cp source          # this is the root of the new class path you created
-cmd neko output.n  # a quick way to run a command after compiling

Notice that you add the -cp flag with the same name as the folder you just created. This can be a relative path to any folder including a parent folder using ellipsis, ../my_parent. You may have also seen the additional -cmd flag which runs the command neko output.n after compiling the program. There is another way to execute neko code using the -x flag which is demonstrated below. Try it and see what happens.

-x Main     # compiles the class Main into a neko binary file named Main.n and executes it
-cp source  # you still need the class path to point to the source folder

Hopefully you can see that the -x flag is a quick way to test a class using the neko target in Haxe.

Multiple Targets

So by now you should have a better grasp on the neko target and some of the flags you can use to develop for neko. What if you also want to compile to javascript as well? One way would be to create another hxml file and build them separately but wouldn't it be better if you could do it all in one command?

-neko output.n
-main Main
-cp source

--next         # tell Haxe to start the next task

-js output.js  # create a javascript file
-main Main
-cp source

You've probably noticed there are two new flags that haven't been used before. The first is --next which tells the Haxe compiler to start a new task which could be a command, another target, or a completely different program. The second flag is -js which will tell the Haxe compiler to build a javascript file.

This works fine but you might not want so much duplication between the two targets. Wouldn't it be nice if there was a way to specify the -main and -cp flags globally for the hxml file? There is!

-main Main
-cp source

--each           # use the flags above for every task below

-neko output.n

--next

-js output.js

--next

-swf out.swf     # compile to a flash swf file
-swf-header 320:240:60:FFFFFF # defines properties for the swf file (width:height:fps:background color)

That's better. There is even a new target added just for fun. So you can see that the --each flag uses all of the flags above it for all of the tasks below. It's a bit like copying and pasting the flags multiple times.

Defines and Debug

Errors are bound to happen and trying to figure out where something broke can be a frustrating process. Thankfully Haxe has included stack traces to help you figure out what went wrong. There is one flag that will help improve stack traces while developing and that is -debug.

Error.hx

class Error {
    public static function main() {
        throw "This is an error";
    }
}

test-error.hxml

-x Error
-debug

Now if you build this code with haxe test-error.hxml you can see that Haxe dumps a stack trace where the throw is. If you were to remove this flag you'll see that the stack trace still occurs but doesn't have as much useful information. When building a release version you will want to remove this flag.

So that's all well and good but what if you want to compile something differently based on a defined value? You can do that with the -D flag. Here is an updated version of the Error class that handles a user defined value.

Error.hx

class Error {
    public static function main() {
        #if mydefine
        trace("this only happens if -D mydefine is included");
        #end
        #if debug
        trace("I'm in debug mode");
        #end
        throw "This is an error";
    }
}
-x Error
-debug
-D mydefine

You might have noticed that there is a condition for a debug define. By using the -debug flag it also automatically includes -D debug as well. You can read more about conditional compilation in the Haxe manual.

External Libraries

Up until now you've only been using the standard library for Haxe. What about adding external code from haxelib? There's a flag for that too! First you should install a library through haxelib.

haxelib install format

Now you are ready to compile with the format library.

-x Main      # compile and execute the Main class for neko
-cp source
-lib format  # include the format library from haxelib

Were you expecting something more difficult? Hopefully this is all making sense and you are ready to move on to the next compile target, C++. But first you should learn about reducing the size of your compiled output.

Dead Code Elimination

When compiling to javascript you may notice that the output can get pretty large. By default Haxe includes a lot of unused functions and classes based on what you import. This is a good thing because it prevents your code from crashing if a class isn't included in the output. However, Haxe is very smart in that it knows what code can be tossed if a function is never called.

-x Main
-cp source
-dce full   # can be "no", "std", or "full"

The new flag is -dce which has a few options. The first is to pass "no", which is the default, and simply doesn't perform dead code elimination. The second is "std" which will strip any unused code in the Haxe standard library. The final one is "full" which eliminates all unused code from your compiled output.

One thing to note however is that Haxe can sometimes remove functions that you need if you are using fancy tricks like reflection. There may also be libraries that do not work well with dead code elimination so be careful that you test thoroughly when this flag is turned on. You may want to read more about how to retain functions and classes in the Haxe manual on dead code elimination.

C++

haxelib install hxcpp

hxcpp is the required library to compile C++ code. It contains a handful of tools and a way to add native extensions with CFFI. Here's an example of how you can compile to C++.

-cpp out    # slightly different than other targets because this defines a directory
-main Main
-cp source

This looks familiar, right? The -cpp flag works almost identical to other targets but instead of naming an output file it names the output directory. If you run haxe build.hxml now you may get an error if no compiler is found on your computer. You can either go through the step to remedy that or use the next method.

Cppia

As of writing this Cppia, pronounced "sepia", is a new feature for Haxe and hxcpp. It is a very quick way to compile C++ code and runs equal to or faster than Neko. That said there are still some early issues especially with null values. So how do you compile for Cppia?

-cpp out.cppia  # unlike before this is actually a file name and not a directory
-D cppia        # this is where the magic happens
-main Main
-cp source
-cmd haxelib run hxcpp out.cppia  # execute the cppia file

There are a few differences you might see. The first is that the -cpp flag no longer takes a directory name but instead it takes a file name. The second is that you need to define a value to tell hxcpp to output Cppia instead of C++. Finally there is a command flag at the bottom which executes the code by passing the newly created cppia file to hxcpp.

Where to go from here

There are a lot of flags that were not covered in this post. If you want to find out what they are the first thing to do is to run haxe -help which will return a list of all the supported flags in Haxe. There is also some useful information in the Haxe manual that may be helpful.

Posted in Haxe

Tags: hxml, hxcpp, cppia, neko


No Comments

OpenGL Resources

OpenGL Resources

OpenGL has gone through several verions over the years but it seems that many of the tutorials you'll find on the internet are dated and mostly cover the fixed function pipeline. I wanted to find more information about programable shaders (glsl), framebuffers, vertex buffers, and many of the other newer features found in later versions of OpenGL. So I've compiled a list of good tutorials and sites I've found that cover these topics.

OpenGL Tutorials

http://open.gl/

Start here first. These tutorials are well written and clearly explain some of the details of OpenGL that seemed mystic to me before. I actually understand the difference between a framebuffer and renderbuffer now!

http://learnopengl.com/

Another great site that covers a wide variety of topics including lighting, shadows, and parallax/normal maps.

http://www.opengl-tutorial.org/

These tutorials include the full source code which is very useful for tinkering.

http://antongerdelan.net/opengl/

Anton Gerdelan has several tutorials on his website as well as a book. He has a good overview of ray picking and cube maps.

Heroku's GLSL

This site is full of mind blowing fullscreen glsl effects. Everything runs in the browser and the source code live reloads. It even lets you tweak anyone's code and save your own changes.

http://glsl.heroku.com/

OpenGL Documentation

OpenGL 4 Reference Pages

http://www.opengl.org/sdk/docs/man/

When you've exhausted the above sites this is the official reference for OpenGL. It has complete descriptions on all of the gl functions. There are probably many others but these gems have helped me immensely. Let me know in the comments if there are other useful sites I missed.

API Docs for multiple versions

http://docs.gl/

This is a great site to get information on different versions of gl* function calls.

Posted in Programming

Tags: OpenGL, tutorials, glsl


No Comments

Haxe Networking and Bonjour

Haxe Networking and Bonjour

I've been working on a party game for the last few months that requires local networking between iOS and Android. The only problem is that I didn't want users to have to type in ip addresses. Enter the world of zero configuration networking...

Bonjour has been around for quite a while and primarily works on iOS and OSX, although Apple has provided a Windows library as well. Essentially a program publishes a services over the network that is available to find from any computer on that network. Another running program can then broadcast out a request to find certain types of services and resolve them. If none are found, the program can choose to stop looking or send out another request.

This is all well and good but Android doesn't have support for Bonjour. Instead someone created jmDNS which is a Java implementation that can talk with Bonjour services, with some minor differences. So now I knew which libraries to use but neither one of them works with Haxe which is the programming language I prefer.

With some help from hxcpp I was able to glue the two libraries together into a single class that can be used on multiple platforms. This solved my problem of finding devices over the network but what about connecting them together?

Haxe has basic socket connection classes, UDP will be in 3.1.0, but no major networking library (that I'm aware of). Using Python's twisted framework as inspiration I decided to create hxnet. It has simple Client/Server code for both TCP and UDP as well as basic protocol classes for RPCs and Telnet.

The Bonjour class is included in hxnet and it should be flexible enough to add protocols like HTTP, FTP, etc... I'm not sure I'd consider it production level code yet but it's solid enough for me to build my game with it. If you want to give it a shot the code is on GitHub and feel free to create pull requests if you find bugs or code up a crazy new feature.

Posted in General

Tags: Haxe, networking, twisted, udp, tcp, hxnet


No Comments

Recently Changed Files in git

When setting up a deployment script from git I needed to know what files were recently changed. Here is a useful script that does exactly that. Just change HEAD~4 to whatever commit you want to go back to.

git log --pretty=oneline --name-only HEAD~4..HEAD | sed -E -e '/[0-9a-zA-Z]{40} /d' | sort | uniq

Posted in General

Tags: git, deployment, script