Comments (4)
Hi @EternalSaga,
Thanks for the script. I tried to put your commands into a bash script like:
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
#git clone https://github.com/FindDefinition/cumm.git $SCRIPT_DIR/cumm
export CUMM_CUDA_VERSION=11.4 # cuda version, required but only used for flag selection when build libspconv.
export CUMM_DISABLE_JIT=1
export SPCONV_DISABLE_JIT=1
export CUMM_INCLUDE_PATH=$SCRIPT_DIR/cumm/include # if you use cumm as a subdirectory, you need this to find cumm includes.
export CUMM_CUDA_ARCH_LIST="8.6" # cuda arch flags
pip3 install wheel pccm -i https://pypi.tuna.tsinghua.edu.cn/simple
cd $SCRIPT_DIR/cumm && git checkout v0.3.7 && python3 setup.py bdist_wheel # cumm has already been installed
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
cd $SCRIPT_DIR/spconv && python3 setup.py bdist_wheel && python3 -m spconv.gencode --include=./cpp/include --src=./cpp/src && \
cd cpp/ && cd src && cmake -DCMAKE_CXX_FLAGS="-I/builds/libspconv/spconv/cpp/include -L/usr/local/cuda/lib64/" . -B ../build && \
cmake --build ../build --config Release -j 8
but got the following error about accessing the header files (though they do exist). Have you ever encountered such issues? I suppose there would be some additional settings in CMakeLists.
-- Configuring done
-- Generating done
-- Build files have been written to: /builds/libspconv/spconv/cpp/build
[ 1%] Building CXX object spconvlib/cumm/conv/main/ConvMainUnitTest/CMakeFiles/spconvlib_cumm_conv_main_ConvMainUnitTest.dir/ConvMainUnitTest_extract_mnk.cc.o
[ 2%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/StaticAllocator/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_StaticAllocator.dir/StaticAllocator_StaticAllocator.cc.o
[ 2%] Building CXX object spconvlib/cumm/gemm/main/GemmMainUnitTest/CMakeFiles/spconvlib_cumm_gemm_main_GemmMainUnitTest.dir/GemmMainUnitTest_get_all_algo_desp.cc.o
[ 3%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/ExternalAllocatorGuard/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_ExternalAllocatorGuard.dir/ExternalAllocatorGuard_ExternalAllocatorGuard.cc.o
[ 4%] Building CXX object spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator/CMakeFiles/spconvlib_spconv_csrc_sparse_alloc_ExternalAllocator.dir/ExternalAllocator_zeros_guard.cc.o
[ 4%] Building CXX object spconvlib/spconv/csrc/sparse/convops/GemmTuneResult/CMakeFiles/spconvlib_spconv_csrc_sparse_convops_GemmTuneResult.dir/GemmTuneResult_is_valid.cc.o
[ 5%] Building CXX object spconvlib/cumm/common/CompileInfo/CMakeFiles/spconvlib_cumm_common_CompileInfo.dir/CompileInfo_get_compiled_cuda_arch.cc.o
[ 6%] Building CXX object spconvlib/spconv/csrc/sparse/convops/gemmops/GemmTunerSimple/CMakeFiles/spconvlib_spconv_csrc_sparse_convops_gemmops_GemmTunerSimple.dir/GemmTunerSimple_GemmTunerSimple.cc.o
/builds/libspconv/spconv/cpp/src/spconvlib/cumm/common/CompileInfo/CompileInfo_get_compiled_cuda_arch.cc:1:10: fatal error: spconvlib/cumm/common/CompileInfo.h: No such file or directory
1 | #include <spconvlib/cumm/common/CompileInfo.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/builds/libspconv/spconv/cpp/src/spconvlib/cumm/conv/main/ConvMainUnitTest/ConvMainUnitTest_extract_mnk.cc:1:10: fatal error: spconvlib/cumm/conv/main/ConvMainUnitTest.h: No such file or directory
1 | #include <spconvlib/cumm/conv/main/ConvMainUnitTest.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/builds/libspconv/spconv/cpp/src/spconvlib/cumm/gemm/main/GemmMainUnitTest/GemmMainUnitTest_get_all_algo_desp.cc:1:10: fatal error: spconvlib/cumm/gemm/main/GemmMainUnitTest.h: No such file or directory
1 | #include <spconvlib/cumm/gemm/main/GemmMainUnitTest.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/alloc/StaticAllocator/StaticAllocator_StaticAllocator.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/alloc/StaticAllocator.h: No such file or directory
1 | #include <spconvlib/spconv/csrc/sparse/alloc/StaticAllocator.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator/ExternalAllocator_zeros_guard.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator.h: No such file or directory
1 | #include <spconvlib/spconv/csrc/sparse/alloc/ExternalAllocator.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
/builds/libspconv/spconv/cpp/src/spconvlib/spconv/csrc/sparse/convops/GemmTuneResult/GemmTuneResult_is_valid.cc:1:10: fatal error: spconvlib/spconv/csrc/sparse/convops/GemmTuneResult.h: No such file or directory
1 | #include <spconvlib/spconv/csrc/sparse/convops/GemmTuneResult.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
from spconv.
I found out that I forgot to prepare spconv/CMakeLists.txt. It works like a charm with this repo's example:
cmake_minimum_required(VERSION 3.20)
project(spconv LANGUAGES CXX CUDA)
include_directories(/builds/libspconv/include)
add_subdirectory(src)
# tell parent spconv2 include path.
set(SPCONV2_INCLUDE_PATH /builds/libspconv/spconv/include PARENT_SCOPE)
install (TARGETS spconv
ARCHIVE DESTINATION lib
LIBRARY DESTINATION lib
RUNTIME DESTINATION bin)
from spconv.
@barrydoooit
Thank you for your suggestion. I didn't notice that there is a CMakeLists.txt in this library. I use the cmake cli arguments ("-DCMAKE_CXX_FLAGS="-I/workspace/spconv/cpp/include -L/usr/local/cuda/lib64/") as a workaround. If you use my solution, please make sure you use the correct include directory path and the cuda library path.
from spconv.
A little bit improvements with barrydoooit's suggestion.
FROM nvidia/cuda:12.2.2-cudnn8-devel-ubuntu22.04 AS basebuild
ENV TZ=Asia/Shanghai
ENV DEBIAN_FRONTEND=noninteractive
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt update && apt install -y lsb-release wget software-properties-common gnupg
RUN sed -i 's/archive.ubuntu.com/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list
RUN apt update && apt install lsb-release wget software-properties-common gnupg -y && apt update && apt upgrade -y && apt install -y vim wget gdb pkg-config autoconf automake build-essential git libpoco-dev uuid-dev libncurses5-dev python3-dev python3-pip \
libtool libc6 libc6-dev unzip libnuma1 libnuma-dev libturbojpeg0-dev apt-utils \
&& python3 -m pip install -i https://pypi.tuna.tsinghua.edu.cn/simple --upgrade pip
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1A127079A92F09ED && apt-add-repository 'deb https://apt.kitware.com/ubuntu/ jammy main' && apt update && apt install -y cmake
WORKDIR /workspace
FROM basebuild as build_stage
RUN git clone https://github.com/traveller59/spconv.git && git clone https://github.com/FindDefinition/cumm
# set environmnet variables for building cumm
ENV CUMM_CUDA_VERSION=12.2
ENV CUMM_DISABLE_JIT=1
ENV SPCONV_DISABLE_JIT=1
ENV CUMM_INCLUDE_PATH=/workspace/cumm/include
ENV CUMM_CUDA_ARCH_LIST="8.9"
# install pccm for building cumm wheel
RUN pip3 install wheel pccm -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN cd /workspace/cumm && git checkout v0.5.3 && python3 setup.py bdist_wheel && pip3 install dist/cumm_cu122-0.5.3-cp310-cp310-linux_x86_64.whl
# remove cumm depandance for building spconv
COPY pyproject.toml /workspace/spconv/pyproject.toml
ENV PATH=/usr/local/cuda-12.2/bin${PATH:+:${PATH}}
ENV LD_LIBRARY_PATH=/usr/local/cuda-12.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
RUN cd /workspace/spconv && mkdir ./cpp
# from example/libspconv/spconv/CMakeLists.txt
COPY CMakeLists.txt /workspace/spconv/cpp/CMakeLists.txt
RUN cd /workspace/spconv && python3 setup.py bdist_wheel && python3 -m spconv.gencode --include=./cpp/include --src=./cpp/src
RUN cd /workspace/spconv/cpp/ && mkdir build && cmake . -B ./build && \
cmake --build ./build --config Release --parallel $(nproc) --target install
RUN mkdir -p ./libspconv/lib && mkdir -p ./libspconv/include
# make sure the cumm headers have been moved out with spconv heads
RUN mv /usr/local/lib/libspconv.so ./libspconv/lib && mv /workspace/spconv/cpp/include ./libspconv/ && mv /workspace/cumm/include/tensorview ./libspconv/include/
FROM basebuild
RUN apt install -y gcc-12 g++-12
RUN wget https://apt.llvm.org/llvm.sh && chmod +x ./llvm.sh && ./llvm.sh 18 -s && rm ./llvm.sh
COPY --from=build_stage /workspace/libspconv /usr/local/libspconv
from spconv.
Related Issues (20)
- Gaussian low-pass filtering
- Point2Voxel generated coordinates out of range? HOT 1
- How can I modified the rule when building the rulebook of SubMConv3d in the latest version of the code?
- Questions about performance HOT 9
- torch.compile(model) in pytorch 2.x
- Changing the num_act_in in Native Algo leads to cuda Illegal address in the example main
- Why there is not spconv.pytorch after install spconv?
- How to get FLOPs when using spconv 2.x? HOT 2
- ERROR: No matching distribution found for spconv-cu114 HOT 1
- import spconv.pytorch will slow down torch.multiprocessing.Pool().apply_async()
- How to visualise spconv HOT 3
- About the usage of sparse_add
- Really Annoying for trying to understand spconv2d, will you please give us a detailed example on the input of [4,3,128,128] (NCHW) up channel to (4,16,128,128) )
- How to Build backward for Sparse tensor dot Multiplication
- spconv could not take advantage of gpu acceleration
- Discrepancy Between Sparse and Dense Convolution Outputs for Density=1.0
- Data exceed int32 range HOT 1
- SparseGlobalAvgPool seems not working
- May I help with maintaining PyPI of spconv and cumm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spconv.