Cannot build on Windows

[amzn]

Cannot build on Windows

[louisharris]

Hey @jawilson, this gstreamer plugin was not initially intended for use on Windows, however theres no reason this should not be able to be built on Windows. I've tried replicating your build steps and I'm also running into errors, but I'm not able to replicate the exact build error message you've shown here yet (I'm getting filepath parsing issues instead). I'll try get around these issues and get back to you soon once I can reproduce the error.

[jawilson]

Thanks @louisharris led me know if there is anything I can do to help.

[loganek]

Hey @jawilson just out of curiosity, did you build AWS SDK with the same compiler you use for building the GStreamer plugin? C++ doesn't standarize ABI (e.g. mangling can be different across different compilers), so that might be a reason why the linker couldn't find those symbols.

[jawilson]

@loganek yes, everything is being build with MSVC 2019. While different compilers would explain static linking issues, I'm not sure why it's failing to find aws-cpp-sdk-sts.lib at all.

[btgoodwin]

I'm in a similar situation as @jawilson.

When I attempt the first route to building aws-cpp-sdk, it installs pkg-config .pc files that indicate the resulting libraries will be installed in c:\aws-cpp-sdk-all\lib, however they're installed in ../bin. This is what's causing the failure to find the library aws-cpp-sdk-sts.lib, for example.

If you manually patch the affected .pc file(s), you wind up at linker problems similar to the second attempt.

Moreover, the linker starts spitting warnings that the library architecture x64 is different than the target machine type of x86 as it tries to link back to the aws-cpp-sdk and gstreamer libraries. There seems to be something funny there in the detected machine type via meson for this repository, amazon-s3-gst-plugin vs. the aws-cpp-sdk's CMAKE approach that we also might want to reconcile.

[btgoodwin]

Going the second route, building without shared libraries, the .lib files are now installed in lib (vs. bin), however because of the cmake configuration, the package configuration files are no longer generated or installed. I think the only way that second route would have worked is if the installation directory wasn't cleared out before re-building on the second attempt (so there are actually these libraries installed in both bin and lib).

I've tried having meson in this repository use the CMAKE_PREFIX_PATH however according to this, it's still broken on v 0.58.0 (related issue).

Based on that issue then, I think we're stuck letting meson use pkg-config to iron out these dependencies, which I think means patching aws-cpp-sdk (the linked issue aws/aws-sdk-cpp#1658 above) so that the package config files are both valid and produced. A comment from a collaborator on that project has in the last hour confirmed the issue.

[btgoodwin]

When building the aws-cpp-sdk, they recommend setting preprocessor definitions for USE_IMPORT_EXPORT and USE_WINDOWS_DLL_SEMANTICS, so combined with the package config patch, we need to also add the -D options to the initial cmake configuration command. The latter will insert __declspec(dllexport) in front of public APIs which will allow the amazon-s3-gst-plugin to link (reference). The former is a question...I see it referenced in several places but when you attempt to configure it in CMake, it warns USE_IMPORT_EXPORT is unused.

Even with that set though, I still get the following LNK2001 errors:

>meson compile -C build -j 0
ninja: Entering directory `build'
[1/12] Compiling C object src/gsts3elements.dll.p/gsts3sink.c.obj
../src/gsts3sink.c(351): warning C4244: 'function': conversion from 'gsize' to 'guint', possible loss of data
[2/12] Compiling C object tests/check/s3sink.exe.p/s3sink.c.obj
../tests/check/s3sink.c(91): warning C4267: 'function': conversion from 'size_t' to 'guint32', possible loss of data
[6/12] Linking target src/gstawscredentials-1.0.dll
FAILED: src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.pdb
"link"  /MACHINE:x64 /OUT:src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.dll.p/gstawscredentials.cpp.obj "/nologo" "/release" "/nologo" "/DEBUG" "/PDB:src\gstawscredentials-1.0.pdb" "/DLL" "/IMPLIB:src\gstawscredentials-1.0.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-sts.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-core.lib" "c:/gstreamer/1.0/msvc_x86_64/lib/gstreamer-1.0.lib" "c:/gstreamer/1.0/msvc_x86_64/lib/gobject-2.0.lib" "c:/gstreamer/1.0/msvc_x86_64/lib/glib-2.0.lib" "c:/gstreamer/1.0/msvc_x86_64/lib/intl.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
gstawscredentials.cpp.obj : error LNK2001: unresolved external symbol "char const * const Aws::Http::CONTENT_TYPE_HEADER" (?CONTENT_TYPE_HEADER@Http@Aws@@3QBDB)
gstawscredentials.cpp.obj : error LNK2001: unresolved external symbol "char const * const Aws::Http::API_VERSION_HEADER" (?API_VERSION_HEADER@Http@Aws@@3QBDB)
src\gstawscredentials-1.0.dll : fatal error LNK1120: 2 unresolved externals
[7/12] Compiling C++ object src/libmultipartuploader.a.p/gsts3multipartuploader.cpp.obj
ninja: build stopped: subcommand failed.

EDIT: As I picked this apart in aws-cpp-sdk, the USE_WINDOWS_DLL_SEMANTICS is getting set automatically by the platform cmake file, andAWSSDK_INSTALL_AS_SHARED_LIBS is being set to match BUILD_SHARED_LIBS as it should in the included setup_cmake_find_module.cmake for the generated ConfigVesion.cmake files (like AWSSDKConfigVersion.cmake). However, the resulting non-generated AWSSDKConfig.cmake:58 is not evaluating to TRUE for me (I added a message(STATUS ... statement which I do not see in my logs).

[btgoodwin]

Here's the build output I get if I apply the aforementioned patch (which only applies to the cpp package configs):

>meson compile -C builddir -j 0
ninja: Entering directory `builddir'
[2/12] Compiling C object src/gsts3elements.dll.p/gsts3sink.c.obj
../src/gsts3sink.c(351): warning C4244: 'function': conversion from 'gsize' to 'guint', possible loss of data
[4/12] Compiling C object tests/check/s3sink.exe.p/s3sink.c.obj
../tests/check/s3sink.c(91): warning C4267: 'function': conversion from 'size_t' to 'guint32', possible loss of data
[6/12] Linking target src/gstawscredentials-1.0.dll
FAILED: src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.pdb
"link"  /MACHINE:x64 /OUT:src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.dll.p/gstawscredentials.cpp.obj "/nologo" "/release" "/nologo" "/DEBUG" "/PDB:src\gstawscredentials-1.0.pdb" "/DLL" "/IMPLIB:src\gstawscredentials-1.0.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-sts.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-core.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/gstreamer-1.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/gobject-2.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/glib-2.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/intl.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
gstawscredentials.cpp.obj : error LNK2001: unresolved external symbol "char const * const Aws::Http::CONTENT_TYPE_HEADER" (?CONTENT_TYPE_HEADER@Http@Aws@@3QBDB)
gstawscredentials.cpp.obj : error LNK2001: unresolved external symbol "char const * const Aws::Http::API_VERSION_HEADER" (?API_VERSION_HEADER@Http@Aws@@3QBDB)
src\gstawscredentials-1.0.dll : fatal error LNK1120: 2 unresolved externals
[7/12] Compiling C++ object src/libmultipartuploader.a.p/gsts3multipartuploader.cpp.obj
ninja: build stopped: subcommand failed.

In that output we can see the C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-core.lib listed in the link command which does have those two symbols exported according to dumpbin:

Dump of file c:\aws-cpp-sdk-all\bin\aws-cpp-sdk-core.lib

File Type: LIBRARY

     Exports

       ordinal    name
...
                  ?API_VERSION_HEADER@Http@Aws@@3QBDB (char const * const Aws::Http::API_VERSION_HEADER)
...
                  ?CONTENT_TYPE_HEADER@Http@Aws@@3QBDB (char const * const Aws::Http::CONTENT_TYPE_HEADER)

For giggles I tried adding to the top-level meson.build: aws_cpp_sdk_core_dep = dependency('aws-cpp-sdk-core', version : aws_cpp_sdk_req). I then added to the src/meson.build that dependency to the list for the gstawscredentials target. The result was the above linking errors were replaced with this:

>meson compile -C builddir -j 0
ninja: Entering directory `builddir'
[3/12] Compiling C object src/gsts3elements.dll.p/gsts3sink.c.obj
../src/gsts3sink.c(351): warning C4244: 'function': conversion from 'gsize' to 'guint', possible loss of data
[4/12] Compiling C object tests/check/s3sink.exe.p/s3sink.c.obj
../tests/check/s3sink.c(91): warning C4267: 'function': conversion from 'size_t' to 'guint32', possible loss of data
[6/12] Linking target src/gstawscredentials-1.0.dll
FAILED: src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.pdb
"link"  /MACHINE:x64 /OUT:src/gstawscredentials-1.0.dll src/gstawscredentials-1.0.dll.p/gstawscredentials.cpp.obj "/nologo" "/release" "/nologo" "/DEBUG" "/PDB:src\gstawscredentials-1.0.pdb" "/DLL" "/IMPLIB:src\gstawscredentials-1.0.lib" "/LIBPATH:C:/aws-cpp-sdk-all/lib" "aws-cpp-sdk-core.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-sts.lib" "C:/aws-cpp-sdk-all/bin/aws-cpp-sdk-core.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/gstreamer-1.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/gobject-2.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/glib-2.0.lib" "C:/gstreamer/1.0/msvc_x86_64/lib/intl.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
LINK : fatal error LNK1181: cannot open input file 'aws-cpp-sdk-core.lib'
[7/12] Compiling C++ object src/libmultipartuploader.a.p/gsts3multipartuploader.cpp.obj
ninja: build stopped: subcommand failed.

From the above, you can see the /LIBPATH:... argument has now been added and points to where the static libraries are installed, which is not where the aws-cpp-sdk-core.lib resides (it is in bin, as you can see in the later arguments). Checking the aws-cpp-sdk-core.pc file over in lib/pkgconfig and we can see it has a different, unpatched version of the package config file.

I tried manually editing the package config file to point at bin for the shared libraries and lib for its private ones, but that only circled me back to my original state at the start of this comment: (1) the /LIBPATH argument doesn't exist and the .lib files are called out with full paths to their explicit locations.

[nirbheek]

Applying both these PRs will fix this issue:

aws/aws-sdk-cpp#1839

#23

[btgoodwin]

@nirbheek Thank you. Despite being able to compile both the SDK and the plugin, I'm still stuck in that gst-inspect for example states it cannot load the module:

c:\Projects\gstreamer-plugins\amazon-s3-gst-plugin>gst-inspect-1.0.exe s3sink

(gst-inspect-1.0:3352): GStreamer-WARNING **: 14:44:54.177: Failed to load plugin 'C:\gstreamer\1.0\msvc_x86_64\lib\gstreamer-1.0\gsts3elements.dll': 'C:\gstreamer\1.0\msvc_x86_64\lib\gstreamer-1.0\gsts3elements.dll': The specified module could not be found.
No such element or plugin 's3sink'

I've tried setting GST_PLUGIN_PATH both to where I installed it (over next to the rest of the gstreamer plugins) as well as the local build/src path where the DLL was built. All result in the same error that it cannot load the module.

Do you have any suggestions? There were no apparent errors during the build.

[nirbheek]

That usually happens because LoadLibrary is unable to find some DLL. Use something like https://github.com/lucasg/Dependencies to figure out what DLL is missing. I would guess that you haven't added the aws sdk DLL directory to PATH.

[btgoodwin]

Yes, that was the problem (not having the SDK's bin directory in my PATH). Thank you!