Mac OS X Building Instructions

[ancurio]

Mac OS X Building Instructions

[tomyun]

I wrote a Homebrew formula for mkxp. It's not submitted to the official tap, so you should install it directly from my repo:

brew install https://raw.githubusercontent.com/tomyun/homebrew-games/mkxp-20160104/mkxp.rb

Please let me know if you have any build / dependency issue. Somehow I couldn't download the prebuilt OS X binary, so testing / comparison was limited. Suggestions are also welcome.

By the way, does mkxp have a plan to release any official version? It'd be nice to have a tagged release with some reasonable versioning scheme. Homebrew may not accept HEAD-only packages in their games tap.

[Ancurio]

Hi b6i6o6,

this is the current situation: I'm a Linux programmer and use the qmake build system on Linux (that's why it had unfortunately no testing whatsoever on other platforms, until your post at least). When humble bundle picked up my engine to use it in one of their ports, @urkle contributed the cmake project files he used to build mkxp on Linux and Mac. That's why the cmake side is so little documented / tested (and I'm unable to give support for it, sorry).

Unfortunately, I am not sure anymore which version of OpenAL I have used. I think it is OpenAL Soft from http://kcat.strangesoft.net/openal.html. Again, compile it.

If I recall correctly, OSX has its own built-in OpenAL interface, doesn't it? It would be interesting how to access that from qmake. Maybe it's just a -framework OpenAL away?

You have to edit the Makefile to replace "-lGL" by "-framework OpenGL" and "-lboost_program_options" by "-lboost_program_options-mt".

About boost: it unfortunately creates different library names depending on whether it's compiled with multithreading support (-mt) or single threaded, and sometimes it has no suffix at all. Add to that the fact that boost has no pkg-config support, and you get pretty hard to discover libraries =/

About the "-lGL": That looks like a leftover of when GLEW didn't link to libGL correctly on my setup. My fault, shouldn't have committed that to the main repo.

As you can see, the qmake side wasn't ready to deal at all with OSX so far. But if it works for you, we could split the unix scope into macx and linux-g++ to add the framework stuff for example.

[b6i6o6]

Well that's funny that I can't make the project using CMake's Makefile then! ;-)
Yes OpenAL is included in OSX, but it doesn't come with the alext.h file you are referencing (source: http://forum.freegamedev.net/viewtopic.php?f=52&t=1583), so CMake finds OpenAL, generates the Makefile, but then make can't link to the alext.h file.
However, replacing -libopenal in the qmake generated Makefile by -framework OpenAL doesn't produce any linking error so I'm not sure what to think about this.
I'm not sure why it doesn't link to the OpenAL Soft one I have installed in /usr/local/include/AL though, because qmake dit it right.

As for qmake not ready for OSX, it's not as bad as you put it. The -lGl at the top of the pro file in LIBS was wrong and I'm sure you'll fix it, but the only real "problem" is -lboost_program_options. Could it be possible to have qmake do something like "if you can link with mt link it, otherwise link without mt" ?

[b6i6o6]

I just saw you made changes to improve support of CMake on OSX, thank you!
Now it does compile the file that previously needed alext.h. However, I now have the following error using CMake's Makefile:
/Users/Malorne/mkxp/mkxp/binding-mri/binding-mri.cpp:215:2: error: use of undeclared identifier 'rb_funcallv'
rb_funcallv(rb_ary_entry(trace, len-1), rb_intern("gsub!"), 2, args);

And just so you know, I checked CMake output when looking for dependencies, and I am now sure it is linking to OSX OpenAL. I have edited the above instructions and removed the need for OpenAL.

[cremno]

/Users/Malorne/mkxp/mkxp/binding-mri/binding-mri.cpp:215:2: error: use of undeclared identifier 'rb_funcallv'
rb_funcallv(rb_ary_entry(trace, len-1), rb_intern("gsub!"), 2, args);

Fixed by #32.

[b6i6o6]

It is now compiling on OSX both with cmake and qmake.
Great job to both of you and thank you for your support !

Note: I didn't spot the problem before because qmake and cmake require different versions of ruby. Apparently qmake was linking the one with rb_funcallv and cmake the one with rb_funcall2, I remember having to sudo port ruby20 and ruby21 for both cmake and qmake to work but I don't recall which one required which version.

[Ancurio]

However, replacing -libopenal in the qmake generated Makefile by -framework OpenAL doesn't produce any linking error so I'm not sure what to think about this.

I'm not too keen how this framework stuff works on OSX, but I'd wager a guess that you left the include path for AL soft in the Makefile, so alext.h was sourced from there (even though the linking took place against the osx AL).

I'm not sure why it doesn't link to the OpenAL Soft one I have installed in /usr/local/include/AL though, because qmake dit it right.

qmake uses pkg-config to determine the include/link flags for AL, which only soft has support for (pkg-config is really a Linux thing). Cmake presumably defaults to the framework.

As for qmake not ready for OSX, it's not as bad as you put it. The -lGl at the top of the pro file in LIBS was wrong and I'm sure you'll fix it, but the only real "problem" is -lboost_program_options. Could it be possible to have qmake do something like "if you can link with mt link it, otherwise link without mt" ?

Hm, boost is such a pain in the butt. A quick fix would be to have yet another variable "BOOST_LIB_SUFFIX" where one can specify the "-mt".

[b6i6o6]

Wouldn't be possible to have a kind of dynamic linking condition ?
Like "if you can link with -mt link it, else link without -mt" ?
I mean, without the user having to actually know about the "-mt" at all ! ;-)

[openmac]

A couple of months after my previous builds, I decided to start from scratch on Mac OS X 10.6.8.
I have not tried with qmake but CMake needs these changes to work:

• shader/tilemapvx.vert
  should be added to "Setup embedded source" section in CMakeLists.txt.
• MRI is set to 2.0 and therefore Ruby 2.0 is used while it seems the actual code requires Ruby 2.1 for things like rb_ary_new_capa.
  I changed MRI to 2.1 and installed and used Ruby 2.1.
• Even if you do not give "framework -OpenGL" to CMake, SDL2 still includes it, but the problem is the fact that Apple seems to use unconventional names for OpenGL features in gl.h file provided by OpenGL.framework, and therefore compiling mkxp fails with several errors like this:

mkxp/src/gl-fun.h:201: error: ‘PFNGLGENBUFFERSPROC’ does not name a type

I guess this issue should have been present when mkxp used glew, but glew's workarounds for this issue masked it in the past.
I'm not sure if my workaround is reasonable, but it seems that including the gl.h and glext.h files from XQuartz (semi-official package including X11 and other headers and libs useful for building and running Linux apps on OS X) fixes this issue. So after installing XQuartz, I added this line to CMakeLists.txt under the elseif(APPLE) section:

SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I/opt/X11/include")

(/opt/X11/include is the path for XQuartz).

I also had to modify the gl-fun.h from mkxp and add these lines:

#include <GL/gl.h>
#include <GL/glext.h>

right after #define GLFUN_H.
After that it compiles and seems to work as expected, although I have not yet tested this new build extensively.

[Ancurio]

shader/tilemapvx.vert should be added to "Setup embedded source" section in CMakeLists.txt.

My mistake. I actually tested the cmake build, but didn't realize that it included files from my qmake build and as such this issue didn't show itself. Fixed now.

MRI is set to 2.0 and therefore Ruby 2.0 is used while it seems the actual code requires Ruby 2.1 for things like rb_ary_new_capa. I changed MRI to 2.1 and installed and used Ruby 2.1.

Fixed, can be compilable with 2.0 again. Although I'd recommend to use newer ruby than 2.0 regardless.

As for the OpenGL issues, how new is your SDL2 install? What happens if you add #include <SDL_opengl_glext.h> below #include <SDL_opengl.h>?

[Ancurio]

Nvm the last bit about SDL_opengl_glext.h, it should always be included by SDL_opengl.h. You definitely shouldn't need to include an entirely new header just to get the function signature definition. If your SDL2 is a couple months old, it will have the PFNGLGENBUFFERSPROC definition in SDL_opengl.h; if your SDL2 is newer, it will be in SDL_opengl_glext.h. Please check where this definition is for you.

[openmac]

Thanks for the fixes. :)

Nvm the last bit about SDL_opengl_glext.h, it should always be included by SDL_opengl.h. You definitely shouldn't need to include an entirely new header just to get the function signature definition. If your SDL2 is a couple months old, it will have the PFNGLGENBUFFERSPROC definition in SDL_opengl.h; if your SDL2 is newer, it will be in SDL_opengl_glext.h. Please check where this definition is for you.

I use SDL 2.0.1 because 2.0.3 fails to build on OS X 10.6.8.
2.0.1 includes PFNGLGENBUFFERSPROC definition in SDL_opengl.h as you expected.
(I know I'm not doing the smartest thing by compiling on 10.6 itself while I could just target 10.6 on 10.9, but in my previous experiences such builds were less stable and caused odd errors, I'm sure most of them were caused by me having even less experience than now and also that build probably depended on libraries built with clang for later versions of OS X, therefore some libs were built with clang and some with gcc).

[Ancurio]

I just took a look at my family's mac and opened the OpenGL/gl.h there, and I believe I found the root of this entire issue. gl.h there doesn't stop at OpenGL 1.3 definitions (like Mesa on Linux does), but goes as far as 2.1, and as such GL_VERSION_1_5 is defined by it which cancels out the definitions in SDL_opengl.h for buffer objects. That would also explain why icculus recently angrily removed all system gl.h includes from SDL2 headers.

Can you test this patch please: http://pastebin.com/73B8mBwH

[urkle]

so, ideally you should only include the SDL_opengl.h and never the system one.. OR use something like GLEW (this is my preference)

[Ancurio]

so, ideally you should only include the SDL_opengl.h and never the system one.. OR use something like GLEW (this is my preference)

System headers were never included directly, SDL_opengl.h did that. Ie. it was an issue with SDL.

[urkle]

Ahh.. so just need to upgrade SDL2 then..

[openmac]

@Ancurio, I applied your patch, it compiled successfully. :)

@urkle, I'm glad to see a comment by you, I greatly benefitted from your contributions while I first started building on Mac back in January.
That SDL2 change seems to land in 2.0.4 which is not released yet, therefore we would depend on another hg build for a lib, other than physfs.

[Ancurio]

Ok, pushed that patch to master. Let me know if there's anything else.

[cremno]

@Ancurio:

Fixed, can be compilable with 2.0 again. Although I'd recommend to use newer ruby than 2.0 regardless.

By the way, when 2.2.0-preview1 is used, mkxp compiles and a simple graphical hello world script works. Maybe cmake should default to 2.1 like qmake does (okay, here you have to edit the file to use a different version)? Or is the default Mac OS X version (2.0, right?) the reason for this discrepancy?

[Ancurio]

By the way, when 2.2.0-preview1 is used, mkxp compiles and a simple graphical hello world script works. Maybe cmake should default to 2.1 like qmake does (okay, here you have to edit the file to use a different version)? Or is the default Mac OS X version (2.0, right?) the reason for this discrepancy?

Yeah, I actually use ruby git head for a game I'm currently working on because of a bug in 2.1. The reason why cmake's default is 2.0 is completely arbitrary, and that wasn't set by me but urkle who originally contributed the cmake build script (I mostly never use cmake myself, so I don't pay as much attention to it). I agree it can be bumped up to 2.1.

The ruby version being "hard-coded" in qmake is actually something I've thought about before, there's no need for it to be that way, and I'll probably make it configurable via an option in the future, but until now simply editing the .pro from my IDE was simply faster so I didn't bother.

[openmac]

Or is the default Mac OS X version (2.0, right?) the reason for this discrepancy?

OS X 10.6, 10.7 and 10.8 come with Ruby 1.8 while OS X 10.9 comes with both Ruby 2.0 and Ruby 1.8.
mkxp's CMake script always targets Mac OS X 10.6 as the minimum version.
So Ruby 2.0 is only preinstalled on OS X 10.9 and even on that version it does not include a pkg-config .pc file.
If I'm not mistaken, urkle's build of mkxp for HumbleBundle included Ruby inside the mkxp binary, and I always bundle ruby.dylib dynamic library with my ports using mkxp, so as far as I'm concerned the CMake script can default to 2.1.

[carstene1ns]

I also agree it can be bumped up (at least for Arch Linux 😄).
It is possible to specify it on the command line as well, so even when it defaults to 2.1, 2.0 or 2.2 can be used easily: https://github.com/carstene1ns/PKGBUILDs/blob/master/mkxp-git/PKGBUILD#L28

[openmac]

For those who may have experienced audio clipping issues on Yosemite, it is caused by the build of OpenAL bundled with OS X 10.10, it is probably a buffer issue in Audio Streaming, which I have reported to Apple.
As a workaround, I built mkxp against OpenAL Soft 1.15 instead and I can still get a build that works fine on OS X 10.6 up to 10.10.

My October 23 build uses OpenAL Soft 1.15 lib instead of the OS X bundled framework:
https://app.box.com/mkxpmacbuilds

[Keyaku]

Concerning physfs: «Enter it and build physfs like the previous dependencies.»
Considering that physfs does not provide a Configure file, it's not to be built like the previous dependencies.
Now, I installed every dependency needed with Homebrew (qmake in mkxp recoginzed them), but since Homebrew doesn't provide PhysicsFS 2.1, I did the following:

mkdir physfs_build; cd physfs_build         # next to physfs/
cmake ../physfs/ -DPHYSFS_BUILD_WX_TEST=FALSE -DPHYSFS_BUILD_TEST=TRUE -DCMAKE_C_FLAGS_RELEASE= -DCMAKE_CXX_FLAGS_RELEASE= -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/physfs/2.1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_FRAMEWORK=
# I get a "Policy CMP0042 is not set: MACOSX_RPATH is enabled by default." warning
make all install
brew link physfs

This just doesn't work. In fact, this, or actually installing it under /usr (not /usr/local) results in the same, which is, after running qmake in mkxp: Project ERROR: Package physfs not found.
I know I'm doing something wrong, but I can't seem to be able to put my finger on it.

[Ancurio]

@Keyaku Have you verified that pkg-config is able to locate your installation?

pkg-config --print-errors physfs

[Keyaku]

@Ancurio Huh… It isn't locating it. In fact, it's not even locating the Homebrew one. Should've noticed this since it doesn't use (or install even) physfs.pc at all. What's the best approach to fix this?

[Ancurio]

pkg-config uses the PKG_CONFIG_PATH environment variable to find all available *.pc files. Try

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig

and try finding the physfs package again, or just running qmake. Homebrew's package is probably not found because they ship the last stable version (very old, 2012) which didn't yet have pkgconfig support.

[Keyaku]

EDIT: I picked the extra/physfs.pc.in and modified it myself (very little effort was needed). qmake now passes that test.

I know, what I'm saying is that the PhysFS build doesn't come with *.pc files in the latest dev release, even though there's definitely a physfs.pc.in in extras/. Is there a flag I'm missing in the build?
I'm sorry for the newb-ish behavior.
And yes, you're right: The last version from Homebrew is 2.0.3, which doesn't have pkgconfig support.

[Ancurio]

Oh, I see now, yes that is actually my fault. Can you download this patch and apply it with patch CMakeLists.txt pkgconfig_osx.patch?

(When I added pkgconfig support to PhysFS I knew very little about OSX development and never considered people would be using it like a regular Unix system..)

[Keyaku]

@Ancurio No worries, and thanks, the patch also made sure the pc file would be properly installed!

[sorlok]

EDIT: Ummm.... what happened to your original question? o_O

Looks like you don't have boost::program_options installed (or linking)
correctly.

Note that boost::program_options is one of boost's few non-header-only
libraries. So you'll need something like:
-lboost_program_options

...in your gcc command. (The other boost libraries are header-only, so they
don't need corresponding -l tags.)

Can you provide the output of:
VERBOSE=1 make

Seth

On Sat, Jun 20, 2015 at 8:37 PM, Keyaku [email protected] wrote:

Sorry to bother again: The process stops at linking, claiming the usual
"Undefined symbols" error, and the problem seemingly coming from Boost (I'm
using 1.58.0). Should I be using any specific build of Boost?

If it helps, here's the output in a Pastie
http://pastie.org/pastes/10250910/text.

—
Reply to this email directly or view it on GitHub
#30 (comment).

[Keyaku]

Here's the VERBOSE=1 output, after using -lboost_program_options.

@sorlok I deleted my original question because I thought I had fixed the linking problems, and so I decided to just discard it (therefore, deleting the question). Unfortunately, it didn't work, so, here I am again :D

[sorlok]

The -lboost_program_options has to go after your objects (c code). Hint:
put it with the other "-l" options, like "-lpthread".

The reason for this is that the linker will only include symbols it knows
that it needs, so (somewhat confusingly) you have to include the object
that needs the symbols before the one that provides it.

Seth

On Sat, Jun 20, 2015 at 8:59 PM, Keyaku [email protected] wrote:

Here's the VERBOSE=1 http://pastie.org/pastes/10250932/text, after
using -lboost_program_options.

I deleted my original question because I thought I have fixed he linking
problems, therefore I decided to just discard it. Unfortunately, it didn't
work, so, here I am again :D

—
Reply to this email directly or view it on GitHub
#30 (comment).

[Keyaku]

@sorlok Ah, sure. Forgot the linker was this picky.
Nonetheless, it didn't work.
Still the same problem. I'm most probably doing it wrong, though. I hope I'm just not being completely idiotic :(

[sorlok]

Hmmm... not sure what to say, except to double-check that you have the
64-bit version of boost program_options installed? Sorry I couldn't be more
helpful.

On Sat, Jun 20, 2015 at 10:58 PM, Keyaku [email protected] wrote:

@sorlok https://github.com/sorlok Ah, sure. Forgot the linker was this
picky.
Nonetheless, it didn't work.
Still the same problem http://pastie.org/pastes/10251033. I'm most
probably doing it wrong, though. I hope I'm just not being completely
idiotic :(

—
Reply to this email directly or view it on GitHub
#30 (comment).

[carstene1ns]

First, you should verify that you are using the correct library. I see the linker also uses /usr/local/lib/libboost_program_options-mt.dylib with -mt suffix, which was for multi-threading variants of the libraries, but has since been removed. This could indicate your version of boost is rather old (5 years or so).
With a short google search I found this questions on stackoverflow, which deal with a similar issue:
http://stackoverflow.com/questions/8454329/why-cant-clang-with-libc-in-c0x-mode-link-this-boostprogram-options-examp
http://stackoverflow.com/questions/11081818/linking-troubles-with-boostprogram-options-on-osx-using-llvm
http://stackoverflow.com/questions/13554671/linker-error-with-boostprogram-options-and-c11-on-osx
The solution seems to be rebuilding boost, as it is linked against that old libstdc++ library.

[Keyaku]

@carstene1ns After @sorlok suggested that I double-checked I had the 64-bit installed, I uninstalled Brew's Boost and decided to download the source myself and compile that (following closely their instructions). Then I tried building mkxp once again. The lib in use this time is, like you'd expect, /usr/local/lib/libboost_program_options.dylib (without the -mt), and it still gives me the linking error.
I'll take a deep look into the links you've dropped and see what I can collect. If I manage to fix it and if the method's deemed shareable (A.K.A. not a screw up of mine), I'll come back here and share it.

[openmac]

@Keyaku, I second @carstene1ns suggestion about libstdc++ or actually libc++ causing this.
If I remember correctly, the first time I tried to compile mkxp on my Mac I experienced the exact same issue.
In my case the issue was the fact that mkxp was going to get compiled while targeting OS X 10.6 as the minimum supported version, and therefore it used libstdc++, while the boost package installed by brew used libc++.
In order to fix this issue, I compiled boost myself, this time using gcc and libstdc++, although I know now that I could just use clang with libstdc++ as described in here:
http://stackoverflow.com/questions/23666754/default-to-lib-stdlibc-for-clang-on-mac-os-mavericks-with-xcode

[Ancurio]

Maybe it's still worth pointing out www.github.com/OutOfOrder/BoostBuilds

I only use it on Linux but the author of it is the one who also did the initial cmake setup for mkxp on OSX, so it should work well.

[Keyaku]

@openmac I swear I tried that, and I also looked into @carstene1ns 's links and I still can't manage to get Boost::program_options to link. I'm truly questioning my capabilities in putting code or packages together, but I firmly believe I'm trying best.
The pre-compiled stuff from @Ancurio 's link worked (thank you!), so although I finally got mkxp to compile and link, I apparently couldn't set up Boost myself.

[urkle]

lurk and I recently added a new branch to the BoostBuilds repo that has libc++ (OS X 10.7+) versions of some of the boost libraries (instead of the libstdc++). And there is now iOS builds (libc++) in there too for way too many architectures :). I'm actually using those boot builds for another project on all of those platforms, so it is being maintained.

Boost is "fun" to get compiling on OS X :)

[ancestral]

Any chance this could be added as a formula to Homebrew?

[openmac]

@tomyun Nice job!
I have been getting reports of issues with box recently, so I may switch to another service, in the meantime here is the latest build I made back in August 2015:
https://www.mediafire.com/folder/z0ezcpei8cpg7/mkxp_builds_for_Mac

Some suggestions for you:

• If you use cmake for building instead of qmake you can remove QT as one of the dependencies.
• fluid-synth should be optional.
• I suggest you make sure openal-soft is being used instead of OS X's bundled framework by using otool -l mkxp on your binary.
• In my latest builds I bundle a patched ruby which has static zlib support, I followed Ancurio's suggestion in here:
  #82 (comment)

[tomyun]

@openmac Thanks for your feedback!

• Also tried to use cmake, but it didn't work well as qmake did. Probably should put off until we have more tweaks and updates in CMakeLists.txt.
• fluid-synth is now :optional.
• Confirmed /usr/local/opt/openal-soft/lib/libopenal.1.dylib is linked with the binary.
• I'm not sure if this patch is relevant to brewed ruby. ruby22 package seems to have built-in zlib support. mkxp binary is also linked with OS X native /usr/lib/libz.1.dylib. Do you know any game / demo that we can test it?

[Ancurio]

By the way, does mkxp have a plan to release any official version? It'd be nice to have a tagged release with some reasonable versioning scheme. Homebrew may not accept HEAD-only packages in their games tap.

Not in the near future. As long as vital functionality such as font rendering is still broken, it makes little sense to me to start maintaining releases. Besides that, I'd like to focus the little time I spent on the project on writing / fixing code instead of version bureaucracy.

[khiav223577]

I wrote a Homebrew formula for mkxp. It's not submitted to the official tap, so you should install it directly from my repo:

brew install https://raw.githubusercontent.com/tomyun/homebrew-games/mkxp-20160104/mkxp.rb

Please let me know if you have any build / dependency issue. Somehow I couldn't download the prebuilt OS X binary, so testing / comparison was limited. Suggestions are also welcome.

@tomyun
The formula is broken with this error message:
homebrew/versions was deprecated. This tap is now empty as all its formulae were migrated.

Do you have any idea how to fix it?

I tried commenting this line depends_on "homebrew/versions/ruby22" and install ruby manually by brew install [email protected], but still not working, keep raising this error: Project ERROR: ruby-2.2 development package not found.