参考:
https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=compile

安装依赖

更新系统

1
apt update && apt upgrade

安装依赖

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apt-get install -y --no-install-recommends \
build-essential \
cmake \
git \
libgoogle-glog-dev \
libgtest-dev \
libiomp-dev \
libleveldb-dev \
liblmdb-dev \
libopencv-dev \
libopenmpi-dev \
libsnappy-dev \
libprotobuf-dev \
openmpi-bin \
openmpi-doc \
protobuf-compiler \
python-dev \

python-pip 上次在其他机器上安装有点问题
之后我就这样安装了

1
2
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

继续安装依赖

1
2
3
4
pip install \
future \
numpy \
protobuf

debian8我们参考ubuntu 16.04的安装吧

1
2
# for Ubuntu 16.04
apt-get install -y --no-install-recommends libgflags-dev

如果有gpu,有一些额外的步骤

我们是vps,跳过

克隆代码和编译

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# Clone Caffe2's source code from our Github repository
git clone --recursive https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init

# Create a directory to put Caffe2's build files in
mkdir build && cd build

# Configure Caffe2's build
# This looks for packages on your machine and figures out which functionality
# to include in the Caffe2 installation. The output of this command is very
# useful in debugging.
cmake ..

# Compile, link, and install Caffe2
sudo make install

pytorch这个仓库里面已经有caffe2了

如果cmake 提示版本问题

1
2
3
4
5
6
[email protected]:~/pytorch/build2# cmake ..
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
CMake 3.2 or higher is required. You are running version 3.0.2


-- Configuring incomplete, errors occurred!

需要cmake3.2或以上版本,本机apt仓库装的最新版是3.0.2

手动下载了一份

1
2
wget https://cmake.org/files/v3.11/cmake-3.11.1-Linux-x86_64.sh
bash cmake-3.11.1-Linux-x86_64.sh

默认会安装在/root下

因为就暂时用下,就不加环境变量了
直接把cmake那步换成

1
/root/cmake-3.11.1-Linux-x86_64/bin/cmake ..

然后执行编译安装

1
make install

然后开始漫长的编译
运行到90%都没问题
运行到93%的时候终于报错了
我就知道不会这么顺利

1
2
3
4
5
6
7
8
9
10
[ 93%] Linking CXX executable ../bin/mpi_test
/usr/bin/ld: CMakeFiles/mpi_test.dir/mpi/mpi_test.cc.o: undefined reference to symbol '_ZN3MPI8Datatype4FreeEv'
//usr/lib/libmpi_cxx.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
caffe2/CMakeFiles/mpi_test.dir/build.make:89: recipe for target 'bin/mpi_test' failed
make[2]: *** [bin/mpi_test] Error 1
CMakeFiles/Makefile2:3218: recipe for target 'caffe2/CMakeFiles/mpi_test.dir/all' failed
make[1]: *** [caffe2/CMakeFiles/mpi_test.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

google搜索

1
caffe2/CMakeFiles/mpi_test.dir/build.make:89: recipe for target 'bin/mpi_test' failed

打开了几个caffe2的issue发现都ref到 #2144

1
Duplicate of #2144

于是尝试https://github.com/caffe2/caffe2/issues/2144#issuecomment-371973437

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
[email protected]:~/pytorch/build# /root/cmake-3.11.1-Linux-x86_64/bin/cmake .. -DUSE_CUDA=OFF -DUSE_MPI=OFF
-- Does not need to define long separately.
-- std::exception_ptr is supported.
-- NUMA is available
-- Turning off deprecation warning due to glog.
-- Current compiler supports avx2 extention. Will build perfkernels.
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/root/pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- The BLAS backend of choice:Eigen
-- Brace yourself, we are building NNPACK
-- Found PythonInterp: /usr/bin/python (found version "2.7.9")
-- Caffe2: Cannot find gflags automatically. Using legacy find.
-- Caffe2: Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Caffe2: Cannot find glog automatically. Using legacy find.
-- Caffe2: Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Failed to find LLVM FileCheck
-- git Version: v0.0.0
-- Version: 0.0.0
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found lmdb (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy (include: /usr/include, library: /usr/lib/libsnappy.so)
-- Found Numa (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
-- OpenCV found (/usr/share/OpenCV)
CMake Warning at cmake/Dependencies.cmake:293 (find_package):
By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Eigen3", but
CMake did not find one.

Could not find a package configuration file provided by "Eigen3" with any
of the following names:

Eigen3Config.cmake
eigen3-config.cmake

Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
"Eigen3_DIR" to a directory containing one of the above files. If "Eigen3"
provides a separate development package or SDK, be sure it has been
installed.
Call Stack (most recent call first):
CMakeLists.txt:107 (include)


-- Did not find system Eigen. Using third party subdirectory.
-- Found PythonInterp: /usr/bin/python (found suitable version "2.7.9", minimum required is "2.7")
-- NumPy ver. 1.14.2 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
-- Could NOT find Gloo (missing: Gloo_INCLUDE_DIR Gloo_LIBRARY)
-- GCC 4.9.2: Adding gcc and gcc_s libs to link line
-- Excluding ideep operators as we are not using ideep
-- NCCL operators skipped due to no CUDA support
-- CUDA RTC operators skipped due to no CUDA support
-- Including image processing operators
-- Excluding video processing operators due to no opencv
-- Excluding mkl operators as we are not using mkl
-- MPI operators skipped due to no MPI support
-- Include Observer library
-- Using lib/python2.7/dist-packages as python relative installation path
-- Automatically generating missing __init__.py files.
-- A previous caffe2 cmake run already created the __init__.py files.
CMake Warning at CMakeLists.txt:226 (message):
Generated cmake files are only fully tested if one builds with system glog,
gflags, and protobuf. Other settings may generate files that are not well
tested.


--
-- ******** Summary ********
-- General:
-- CMake version : 3.11.1
-- CMake command : /root/cmake-3.11.1-Linux-x86_64/bin/cmake
-- Git version : v0.1.11-7992-gdf2817d
-- System : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 4.9.2
-- BLAS : Eigen
-- CXX flags : -Wno-deprecated -fvisibility-inlines-hidden -DONNX_NAMESPACE=onnx_c2 -O2 -fPIC -Wno-narrowing -Wno-invalid-partial-specialization
-- Build type : Release
-- Compile definitions :
--
-- BUILD_BINARY : ON
-- BUILD_CUSTOM_PROTOBUF : ON
-- Link local protobuf : ON
-- BUILD_DOCS : OFF
-- BUILD_PYTHON : ON
-- Python version : 2.7.9
-- Python includes : /usr/include/python2.7
-- BUILD_SHARED_LIBS : ON
-- BUILD_TEST : ON
-- USE_ATEN : OFF
-- USE_ASAN : OFF
-- USE_CUDA : OFF
-- USE_EIGEN_FOR_BLAS : 1
-- USE_FFMPEG : OFF
-- USE_GFLAGS : ON
-- USE_GLOG : ON
-- USE_GLOO : ON
-- USE_LEVELDB : ON
-- LevelDB version : 1.17
-- Snappy version : 1.1.2
-- USE_LITE_PROTO : OFF
-- USE_LMDB : ON
-- LMDB version : 0.9.14
-- USE_METAL : OFF
-- USE_MKL :
-- USE_MOBILE_OPENGL : OFF
-- USE_MPI : OFF
-- USE_NCCL : OFF
-- USE_NERVANA_GPU : OFF
-- USE_NNPACK : ON
-- USE_OBSERVERS : ON
-- USE_OPENCL : OFF
-- USE_OPENCV : ON
-- OpenCV version : 2.4.9.1
-- USE_OPENMP : OFF
-- USE_PROF : OFF
-- USE_REDIS : OFF
-- USE_ROCKSDB : OFF
-- USE_ZMQ : OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /root/pytorch/build

再make install
这次

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
[ 99%] Building CXX object binaries/CMakeFiles/make_cifar_db.dir/make_cifar_db.cc.o
[ 99%] Linking CXX executable ../bin/make_cifar_db
[ 99%] Built target make_cifar_db
Scanning dependencies of target split_db
[100%] Building CXX object binaries/CMakeFiles/split_db.dir/split_db.cc.o
[100%] Linking CXX executable ../bin/split_db
[100%] Built target split_db
Scanning dependencies of target convert_caffe_image_db
[100%] Building CXX object binaries/CMakeFiles/convert_caffe_image_db.dir/convert_caffe_image_db.cc.o
[100%] Linking CXX executable ../bin/convert_caffe_image_db
[100%] Built target convert_caffe_image_db
Scanning dependencies of target db_throughput
[100%] Building CXX object binaries/CMakeFiles/db_throughput.dir/db_throughput.cc.o
[100%] Linking CXX executable ../bin/db_throughput
[100%] Built target db_throughput
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/share/cmake/Caffe2/Caffe2ConfigVersion.cmake
-- Installing: /usr/local/share/cmake/Caffe2/Caffe2Config.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/cuda.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/glog.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/gflags.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/protobuf.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/threads.cmake
-- Installing: /usr/local/share/cmake/Caffe2/public/utils.cmake
-- Installing: /usr/local/share/cmake/Caffe2/Caffe2Targets.cmake
-- Installing: /usr/local/share/cmake/Caffe2/Caffe2Targets-release.cmake
-- Installing: /usr/local/lib/libprotobuf-lite.a
-- Installing: /usr/local/lib/libprotobuf.a
-- Installing: /usr/local/lib/libprotoc.a
-- Installing: /usr/local/bin/protoc
-- Installing: /usr/local/lib/pkgconfig/protobuf.pc
-- Installing: /usr/local/lib/pkgconfig/protobuf-lite.pc
-- Installing: /usr/local/include/google/protobuf/any.h
-- Installing: /usr/local/include/google/protobuf/any.pb.h
-- Installing: /usr/local/include/google/protobuf/api.pb.h
-- Installing: /usr/local/include/google/protobuf/arena.h
-- Installing: /usr/local/include/google/protobuf/arena_impl.h
-- Installing: /usr/local/include/google/protobuf/arenastring.h
-- Installing: /usr/local/include/google/protobuf/compiler/code_generator.h
-- Installing: /usr/local/include/google/protobuf/compiler/command_line_interface.h
-- Installing: /usr/local/include/google/protobuf/compiler/cpp/cpp_generator.h
-- Installing: /usr/local/include/google/protobuf/compiler/csharp/csharp_generator.h
-- Installing: /usr/local/include/google/protobuf/compiler/csharp/csharp_names.h
-- Installing: /usr/local/include/google/protobuf/compiler/importer.h
-- Installing: /usr/local/include/google/protobuf/compiler/java/java_generator.h
-- Installing: /usr/local/include/google/protobuf/compiler/java/java_names.h
...
...

完整的安装见 caffe2-make-install-log

好像是装成功了
测试下

1
2
[email protected]:~/pytorch/build# cd ~ && python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
Failure

擦。

ipython中导入

1
2
3
4
5
6
7
In [1]: from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
CRITICAL:root:Cannot load caffe2.python. Error: libcaffe2.so: cannot open shared object file: No such file or directory
An exception has occurred, use %tb to see the full traceback.

SystemExit: 1

https://github.com/caffe2/caffe2/issues/1904

1
2
3
4
5
6
7
8
9
[email protected]:~/pytorch/build# which python2.7
/usr/bin/python2.7
[email protected]:~/pytorch/build# python2.7
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu

然后发现,在pytorch/build目录下就不会报错

1
2
3
4
5
6
7
8
9
10
11
12
13
[email protected]:~/pytorch/build# ipython
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
Type "copyright", "credits" or "license" for more information.

IPython 2.3.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]: from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu

搜索

1
2
[email protected]:~/pytorch# find . -name libcaffe2.so
./build/lib/libcaffe2.so

就是这个

1
2
[email protected]:~/pytorch/build/lib# ls -lh libcaffe2.so
-rwxr-xr-x 1 root root 16M Apr 24 06:44 libcaffe2.so

vim /etc/profile

1
export LD_LIBRARY_PATH="/root/pytorch/build/lib"

source /etc/profile

现在导入就没有报错了

1
2
3
4
5
6
7
8
[email protected]:~# python
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
>>>