Non-existent "libcuda_nvcc.lib" link library added to VS projects with --enable-cuda-nvcc

[shiftmediaproject]

Non-existent "libcuda_nvcc.lib" link library added to VS projects with --enable-cuda-nvcc

[oviano]

Just to follow up on this. I had to take a few manual steps to complete the build with this configuration. Documenting them here in case it helps someone or as an indication of what needs to happen.

1. Manually compile three cuda files; yadif_cuda.cu, scale_cuda.cu and thumbnail_cuda.cu to produce equivalently named PTX files, using NVidia’s nvcc compiler.

2. Copied them to a Mac and ran an older version of the shell script ptx2c.sh found in the FFmpeg directory to produce similarly named .c files. The latest version tries to do this but removes the letter r instead of the \r.

3. Copied the c files back, and used Notepad++ to remove all the \r from the C files. Otherwise the C files wouldn’t compile.

4. Added these c files to the VS avfilter project.

This was all necessary to get rid of the link errors.

I am sure there must be a way to automate all of this......but having zero experience of CUDA files, compiling etc it’s not something I could figure out easily.

[oviano]

The following bash snippet can be used to compile the .cu files prior to calling project_generate.exe, which will then pick up the generated .c files:

#!/bin/bash

for f in $(find . -name '*.cu'); do
  basename="${f%.*}"
  nvcc -ptx -o $basename.ptx $basename.cu || exit 1
  ./compat/cuda/ptx2c.sh $basename.ptx.c $basename.ptx || exit 1
  rm $basename.ptx || exit 1
done

I tested this snippet with the version of bash found with a git installation, but presumably it would work with any Windows bash shell.

It obviously requires the CUDA Toolkit to be installed (for nvcc.exe) and also requires the 64-bit version of cl.exe to be found in PATH, since nvcc.exe uses cl.exe.

This is most easily achieved by:

call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat” x86_amd64

(or similar, according to your VS installation).

Anyway, thought I'd share these steps/this mini script for anyone else wanting to enable CUDA in this project.

[windowsair]

Thank you for your excellent work!

The final .c file is essentially a string. On my vs2019, a memory leak occurred while compiling this string with large length (this seems to be a documented compiler issue) . I converted the string to a hex array and everything works fine!

To help those who come later, I would like to remind you that the filters scale_cuda, yadif_cuda, thumbnail_cuda, overlay_cuda are only included when the compile option -enable-cuda-llvm or -enable-cuda-nvcc is enabled. On windows platforms, just use the above method with -enable-cuda-nvcc.

[Sibras]

Ive fixed the incorrect inclusion of "libcuda_nvcc.lib" in the repo. However adding support for ptx generation and inclusion is alot more complicated (as your list of above steps demonstrates).

Unfortunately ffmpeg doesnt use the cuda runtime api as that would have made it alot easier. So it may take quite some time to work out how to integrate the required steps into the project.

[oviano]

No problem Matthew, and thanks for the fix.

I guess it's something of an edge-case and I've worked around it by just doing the steps manually. I don't think those CU files are likely to change frequently.

I think the way FFmpeg have done it might be to avoid a dependency on the CUDA runtime libs (related to licensing). i.e. previously I think if FFmpeg had been built with the deprecated enable-cuda-sdk then it would have required the runtime CUDA libraries to be installed at the place FFmpeg was to be run, or for them to be distributed with FFmpeg.

Fairly recently this was changed so that it works the same as nvenc/nvdec in that it uses the libraries that come with the graphics drivers.

I don't fully understand it, but that seems to be the gist of it from a bit of googling and reading the FFmpeg mailing lists.

[oviano]

Just updating back here as a few things seem to have changed with latest FFmpeg.

They removed ptx2c.sh, which I was using in my build scripts to produce the ptx.c files (as detailed above), and seem to be doing things slightly differently now.

Anyway, I recovered ptx2c.sh but found it *no longer worked, but here is a working one. This will produce ptx.c files from the .cu files in a format that works with latest FFmpeg.

#!/bin/sh

# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.

set -e

OUT="$1"
IN="$2"
NAME="$(basename "$IN" | sed 's/\..*//')"

printf "const unsigned char ff_%s_ptx_data[] = {\\" "$NAME" > "$OUT"
echo >> "$OUT"
xxd -i < "$IN" >> "$OUT"
echo "  ,0x00" >> "$OUT"
echo "};" >> "$OUT"
printf "const unsigned int ff_%s_ptx_len = sizeof(ff_%s_ptx_data);" "$NAME" "$NAME" >> "$OUT"


exit 0

*it actually caused the MSVC compiler to use up all resources and eventually run out of heap space, while seriously degrading the performance of the machine.