thejinchao/libprotobuf

libprotobuf for Unreal Engine 4

libprotobuf for Unreal Engine 4

Link the google's protocol bufffers library as the third party in Unreal Engine 4.

Version

• Protobuf: 3.19.0
• Unreal Engine 4: 4.27.2

Usage

1. Import or copy the folder libprotobuf into <your project>/Source/ThirdParty/libprotobuf.
2. Add the libprotobuf as a module into <your project>.Build.cs

PrivateDependencyModuleNames.AddRange(new string[] { "libprotobuf" });

Compile proto file(s)

1. Compile the proto files for Unreal Engine 4 by generate_for_ue4.py.
   • python generate_for_ue4.py --proto_input <proto_file_or_path> --cpp_out <output_path>
   • ex: python generate_for_ue4.py --proto_input Message.proto --cpp_out d:\Prject\Source\ProtoFiles
2. Include and use the header file(ex: Message.pb.h) in your .cpp file.

Build Library

The environment variables required for compilation

• PB_LIBRARY_PATH The directory where this document is located

Apply the patch

cd %PB_LIBRARY_PATH%\protobuf-source
git apply ..\build\patch\diff-base-on-3.19.0.diff

1. Windows

Visual Studio 2019 and CMake are required

mkdir %PB_LIBRARY_PATH%\build\_win64 & cd %PB_LIBRARY_PATH%\build\_win64
cmake -G "Visual Studio 16 2019" -A x64 ^
 -DCMAKE_INSTALL_PREFIX=%PB_LIBRARY_PATH%/libprotobuf ^
 -DCMAKE_MSVC_RUNTIME_LIBRARY="MultiThreaded$<$<CONFIG:Debug>:Debug>DLL" ^
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false ^
 -Dprotobuf_DEBUG_POSTFIX="" ^
 -DCMAKE_INSTALL_LIBDIR="lib/win64/$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>" ^
 -DCMAKE_INSTALL_CMAKEDIR=lib/win64/cmake ^
 -Dprotobuf_MSVC_STATIC_RUNTIME=false ^
 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL ^
 %PB_LIBRARY_PATH%/protobuf-source/cmake
cmake --build . --target INSTALL --config Debug
cmake --build . --target INSTALL --config Release

2. Linux(Cross Compiling)

Clang cross-compile toolchain, Ninja and Unreal Engine Source Code are required, and make sure the following environment variables are set correctly

• UE_THIRD_PARTY_PATH UE third party source directory, This directory is located in a subdirectory Engine/Source/ThirdParty of the UE source directory
• LINUX_MULTIARCH_ROOT The directory where you installed Clang cross-compile toolchain for Linux
• NINJA_EXE_PATH The file path of ninja.exe, Ninja is a small build system with a focus on speed.

mkdir %PB_LIBRARY_PATH%\build\_linux & cd %PB_LIBRARY_PATH%\build\_linux
cmake -G "Ninja Multi-Config" -DCMAKE_MAKE_PROGRAM=%NINJA_EXE_PATH% ^
 -DCMAKE_TOOLCHAIN_FILE="%PB_LIBRARY_PATH%\build\linux\ue4-linux-cross-compile.cmake" ^
 -DUE_THIRD_PARTY_PATH=%UE_THIRD_PARTY_PATH% -Dprotobuf_DEBUG_POSTFIX="" ^
 -DCMAKE_INSTALL_PREFIX=%PB_LIBRARY_PATH%/libprotobuf ^
 -DCMAKE_INSTALL_LIBDIR="lib/linux/$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>" ^
 -DCMAKE_INSTALL_CMAKEDIR=lib/linux/cmake ^
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false ^
 -Dprotobuf_BUILD_EXAMPLES=false ^
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false ^
 %PB_LIBRARY_PATH%/protobuf-source/cmake
cmake --build . --target install --config Debug
cmake --build . --target install --config Release

3. Android

Android Studio is required. And you need install other additional sdk and tools through SDK Manager.

• Android NDK
• Android SDK Build-Tools
• Android SDK Command-line Tools
• CMake make sure the following environment variables are set correctly
• ANDROID_HOME The directory where you installed the Android SDK, It must contain a directory called platform-tools.
• NDKROOT The directory where you unzipped the Android NDK, It must contain a file called ndk-build.cmd
• NDK_CMAKE_VERSION The version number of cmake that comes with the Android NDK, like 3.22.1(Must later than 3.18)

mkdir %PB_LIBRARY_PATH%\build\_android & cd %PB_LIBRARY_PATH%\build\_android
for /d %a in (armeabi-v7a arm64-v8a x86_64) do (
mkdir %a & pushd %a ^
 & "%ANDROID_HOME%\cmake\%NDK_CMAKE_VERSION%\bin\cmake.exe" -G "Ninja Multi-Config" ^
 -DCMAKE_TOOLCHAIN_FILE="%NDKROOT%\build\cmake\android.toolchain.cmake" ^
 -DCMAKE_MAKE_PROGRAM=%ANDROID_HOME%\cmake\%NDK_CMAKE_VERSION%\bin\ninja.exe ^
 -DANDROID_ABI=%a -Dprotobuf_DEBUG_POSTFIX="" ^
 -DCMAKE_INSTALL_PREFIX=%PB_LIBRARY_PATH%/libprotobuf ^
 -DCMAKE_INSTALL_LIBDIR="lib/android/%a/$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>" ^
 -DCMAKE_INSTALL_CMAKEDIR=lib/android/%a/cmake ^
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false ^
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false ^
 %PB_LIBRARY_PATH%/protobuf-source/cmake ^
 & "%ANDROID_HOME%\cmake\%NDK_CMAKE_VERSION%\bin\cmake.exe" --build . --target install --config Debug ^
 & "%ANDROID_HOME%\cmake\%NDK_CMAKE_VERSION%\bin\cmake.exe" --build . --target install --config Release ^
 & popd
)

4. PlayStation 4

PS4 SDK(Orbis) is required.

mkdir %PB_LIBRARY_PATH%\build\_ps4 & cd %PB_LIBRARY_PATH%\build\_ps4
"%SCE_ROOT_DIR%\ORBIS\Tools\CMake\PS4CMake.bat" ^
 -DCMAKE_INSTALL_PREFIX=%PB_LIBRARY_PATH%/libprotobuf ^
 -DCMAKE_INSTALL_LIBDIR="lib/ps4/$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>" ^
 -DCMAKE_INSTALL_CMAKEDIR=lib/ps4/cmake -DCMAKE_CXX_STANDARD=14 ^
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false ^
 -Dprotobuf_BUILD_EXAMPLES=false ^
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false ^
 -Dprotobuf_DISABLE_RTTI=true ^
 %PB_LIBRARY_PATH%/protobuf-source/cmake
cmake --build . --target INSTALL --config Debug
cmake --build . --target INSTALL --config Release

5. PlayStation 5

PS5 SDK(Prospero) is required.

mkdir %PB_LIBRARY_PATH%\build\_ps5 & cd %PB_LIBRARY_PATH%\build\_ps5
"%SCE_ROOT_DIR%\Prospero\Tools\CMake\PS5CMake.bat" ^
 -DCMAKE_INSTALL_PREFIX=%PB_LIBRARY_PATH%/libprotobuf ^
 -DCMAKE_INSTALL_LIBDIR="lib/ps5/$<$<CONFIG:Debug>:Debug>$<$<CONFIG:Release>:Release>" ^
 -DCMAKE_INSTALL_CMAKEDIR=lib/ps5/cmake -DCMAKE_CXX_STANDARD=14 ^
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false ^
 -Dprotobuf_BUILD_EXAMPLES=false ^
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false ^
 -Dprotobuf_DISABLE_RTTI=true ^
 %PB_LIBRARY_PATH%/protobuf-source/cmake
cmake --build . --target INSTALL --config Debug
cmake --build . --target INSTALL --config Release

6. Mac

Xcode and CMake are required.

mkdir -p $PB_LIBRARY_PATH/build/_mac && cd $PB_LIBRARY_PATH/build/_mac
cmake -G "Unix Makefiles" \
 -DCMAKE_INSTALL_PREFIX=$PB_LIBRARY_PATH/libprotobuf \
 -DCMAKE_INSTALL_LIBDIR=lib/mac -DCMAKE_OSX_DEPLOYMENT_TARGET=10.14 \
 -DCMAKE_INSTALL_CMAKEDIR=lib/mac/cmake -DCMAKE_CXX_STANDARD=14 \
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false \
 -Dprotobuf_BUILD_EXAMPLES=false \
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false \
 $PB_LIBRARY_PATH/protobuf-source/cmake
cmake --build . --target install --config Release

7. iOS

Xcode and CMake are required.

mkdir -p $PB_LIBRARY_PATH/build/_ios && cd $PB_LIBRARY_PATH/build/_ios
cmake -G "Unix Makefiles" \
 -DCMAKE_INSTALL_PREFIX=$PB_LIBRARY_PATH/libprotobuf \
 -DCMAKE_TOOLCHAIN_FILE=$PB_LIBRARY_PATH/build/ios/ios.toolchain.cmake \
 -DCMAKE_INSTALL_LIBDIR=lib/ios -DPLATFORM=OS64 \
 -DCMAKE_INSTALL_CMAKEDIR=lib/ios/cmake -DCMAKE_CXX_STANDARD=17 \
 -Dprotobuf_BUILD_TESTS=false -Dprotobuf_WITH_ZLIB=false \
 -Dprotobuf_BUILD_EXAMPLES=false \
 -Dprotobuf_BUILD_PROTOC_BINARIES=false -Dprotobuf_BUILD_LIBPROTOC=false \
 $PB_LIBRARY_PATH/protobuf-source/cmake
cmake --build . --target install --config Release

License

Use The MIT License.

Reference

1. https://github.com/code4game/libprotobuf
2. https://unrealcommunity.wiki/standalone-dedicated-server-i5qjfc27
3. https://unrealcommunity.wiki/linking-static-libraries-using-the-build-system-1ahhe4vt
4. https://unrealcommunity.wiki/compiling-for-linux-nutp04d0
5. https://docs.unrealengine.com/4.27/en-US/SharingAndReleasing/Linux/

libprotobuf's Issues

How to compile arm64 lib for Android

How to compile arm64 lib for Android

Some error were encountered while trying to compile protobuf for PS4

Environment:

• UE4.26.2
• Protobuf 3.19.4
• VS2017
• clang tool chain v17 10.0.1-centos7
• cmake 3.22.3
• PS4 SDK 8.000

Encountered following problems：

1. CMakeList.txt specified to build with c++11, however some header files of PS4 SDK have following macro definitions which caused build failure:

#if __cplusplus<=201103L  
  #error This version of C++ is not compatible with the standard library implementation. 
#endif

I made it work by changing the specified c++ version from 11 to 14 in CMakeList.txt

2. Some header files of protobuf have following code which also caused build failure:

#elif defined(__FreeBSD__)  
#include <sys/endian.h>  // __BYTE_ORDER

Compiler can't find this header file because I am building these files on Windows platform. It works after I modified this code block to

#elif defined(__FreeBSD__)
#define PROTOBUF_LITTLE_ENDIAN 1

3. Finally I don't know how to solve following errors because I am not a C++ developer but a golang developer QAQ

G:\protobuf-3.19.4\src\google\protobuf\arena_impl.h(534,44): error : alignment (64) of thread-local variable 'thread_cache' is greater than the maximum supported alignment (32) for a thread-local variable on this target [G:\protobuf-3.19.4\cmake\build\PS4\libprotobuf.vcxproj]

G:\protobuf-3.19.4\src\google\protobuf\stubs\mutex.h(124,13): error : constexpr constructor never produces a constant expression [-Winvalid-constexpr] [G:\protobuf-3.19.4\cmake\build\PS4\libprotobuf.vcxproj]

Package Failed in UE4.27.1 Source code version

I have done what you said. In Editor it works very well. But when I package the game, it gives me errors as follows. But if I delete the libprotobuf folder from ThirdParty it is OK with no error.

UATHelper: Packaging (Windows (64-bit)): Running AutomationTool...
UATHelper: Packaging (Windows (64-bit)): Parsing command line: -ScriptsForProject=E:/Projects/MyProject2/MyProject2.uproject BuildCookRun -nocompileeditor -nop4 -project=E:/Projects/MyProject2/MyProject2.uproject -cook -stage -archive -archivedirectory=E:/Projects/MyProject2 -package -ue4exe=D:\ProgramFiles\UES_427M\Engine\Binaries\Win64\UE4Editor-Cmd.exe -c
ompressed -ddc=DerivedDataBackendGraph -pak -prereqs -nodebuginfo -targetplatform=Win64 -build -clientconfig=Development -utf8output -compile
UATHelper: Packaging (Windows (64-bit)): Dependencies are out of date. Compiling scripts....
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\VisualStudio2019\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2203,5): warning MSB3245: 未能解析此引用。未能找到程序集“Windows”。请检查磁盘上是否存在该程序集。 如果您的代码需要此引用，则可能出现编译错误。 [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(17,15): error CS0234: 命名空间“Windows”中不存在类型或命名空间名“Management”(是否缺少程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,24): error CS0246: 未能找到类型或命名空间名“IAsyncOperationWithProgress<,>”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,52): error CS0246: 未能找到类型或命名空间名“DeploymentResult”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,70): error CS0246: 未能找到类型或命名空间名“DeploymentProgress”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): Took 1.7939787s to run MSBuild.exe, ExitCode=1
UATHelper: Packaging (Windows (64-bit)): ERROR: Failed to build "D:/ProgramFiles/UES_427M/Engine/Programs/AutomationTool/Saved\UATTempProj.proj":
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\VisualStudio2019\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2203,5): warning MSB3245: 未能解析此引用。未能找到程序集“Windows”。请检查磁盘上是否存在该程序集。 如果您的代码需要此引用，则可能出现编译错误。 [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(17,15): error CS0234: 命名空间“Windows”中不存在类型或命名空间名“Management”(是否缺少程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,24): error CS0246: 未能找到类型或命名空间名“IAsyncOperationWithProgress<,>”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,52): error CS0246: 未能找到类型或命名空间名“DeploymentResult”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)): D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLensPlatform.Automation.cs(1235,70): error CS0246: 未能找到类型或命名空间名“DeploymentProgress”(是否缺少 using 指令或程序集引用?) [D:\ProgramFiles\UES_427M\Engine\Source\Programs\AutomationTool\HoloLens\HoloLens.Automation.csproj]
UATHelper: Packaging (Windows (64-bit)):
UATHelper: Packaging (Windows (64-bit)): (see D:\ProgramFiles\UES_427M\Engine\Programs\AutomationTool\Saved\Logs\Log.txt for full exception trace)
PackagingResults: Error: Failed to build "D:/ProgramFiles/UES_427M/Engine/Programs/AutomationTool/Saved\UATTempProj.proj":
UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=1 (Error_Unknown)
UATHelper: Packaging (Windows (64-bit)): BUILD FAILED
PackagingResults: Error: Unknown Error

Compile protobuf with Visual Studio 2017 and integrate with UE4

Hi,

I already compiled protobuf with visual studio 2017 ( static compile in 64 bits)
To validate the result I have developed a simple console application to check if I can use the *.pb.cc and *.pb.h files
Everything is ok

Next step I tried to incorporate in a unreal engine 4 project. Set up a build files

File.Build.cs

...
PublicIncludePaths.Add(ModulePath + "/Include");
PublicLibraryPaths.Add(ModulePath + "/Lib");

    PublicAdditionalLibraries.Add("libprotobuf.lib");

...

But it gave me strange compile errors

1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.11.25503\INCLUDE\type_traits(565): error C4647: behavior change: __is_pod(google::protobuf::internal::AuxillaryParseTableField) has different value in previous versions

...\Include\google/protobuf/generated_message_table_driven.h(159): note: see reference to class template instantiation 'std::is_podgoogle::protobuf::internal::AuxillaryParseTableField' being compiled

...\Messages.pb.h(992): warning C4668: 'PROTOBUF_INLINE_NOT_IN_HEADERS' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
...\Include\google/protobuf/descriptor.pb.h(4113): warning C4668: 'PROTOBUF_INLINE_NOT_IN_HEADERS' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'
...\Messages.pb.cc(258): error C4125: decimal digit terminates octal escape sequence
...\Messages.pb.cc(259): error C4125: decimal digit terminates octal escape sequence

Search on web but I haven't mutch success: Example this error is real strange !!!!

"...\MSVC\14.11.25503\INCLUDE\type_traits(565): error C4647: behavior change: __is_pod(google::protobuf::internal::AuxillaryParseTableField) has different value in previous versions .."

Have you tried to compile this source code with recent versions of Protobuf and Visual Studio ?

Protocol Buffer 3.3.0
Visual Studio 2017
Release 64 bits

Regards
Nelson Bilber