Giter Site home page Giter Site logo

homebrew-dmc's Introduction

Homebrew Data Management clients

How do I install these formulae?

You need first to install the Homebrew tool. Then,

brew install cern-fts/dmc/<formula>

Or brew tap cern-fts/dmc and then brew install <formula>.

Example

brew tap cern-ft/dmc
brew install srm-ifce
gfal_srm_ifce_version

gfal2-python and gfal2-util

There are two options for installing these two components. They can be either installed via Homebrew, as the rest, or inside a Python virtualenv.

Homebrew

brew install gfal2-python
brew install gfal2-util

Note that there is no need to install all components manually. If you are just interested on gfal2-util, you can just do brew install gfal2-util and all the required dependencies will be downloaded, built and installed.

virtualenv

$ sudo easy_install pip
$ sudo pip install virtualenv
$ brew install gfal2
$ brew install boost-python
$ virtualenv gfal2
$ source gfal2/bin/activate
(gfal2) $ pip install "git+https://gitlab.cern.ch/dmc/gfal2-bindings.git@develop"
(gfal2) $ python -c 'import gfal2; print gfal2.__version__'
1.8.3
(gfal2) $ pip install "git+https://gitlab.cern.ch/dmc/gfal2-util.git@develop"
(gfal2) $ gfal-ls "file:///"
...

Help! I can't connect to secure endpoints

To connect to Grid endpoints normally a proxy is required, sometimes with the proper VO extensions. Additionally, these endpoints will present the client application a X509 certificate that prooves they are who they say they are. These certificates are signed by a trusted Certification Authority, whose certificates need to be present in the client machine.

So, basically, we need to fix two related issues

Install the trusted CA's

You will need to grab the required root certificates from the EGI Repository. For instance, if we want to access endpoints signed by CERN, you will need ca_CERN-GridCA-1.65.tar.gz and ca_CERN-Root-2-1.65.tar.gz.

Anyway, just for facilitate the setup, let's install all of them:

brew install wget
sudo su -
mkdir -p /etc/grid-security/certificates
cd /etc/grid-security/certificates/
wget -r -l1 --no-parent -nd --accept=.tar.gz "http://repository.egi.eu/sw/production/cas/1/current/tgz/"
for i in `ls *.tar.gz`; do tar xzf $i --strip-components=1; done
rm -vf robots.txt *.tar.gz

Now the Root certificates are installed under /etc/grid-security/certificates, where the tools can find them.

You can validate it works running voms-proxy-init (without --voms yet!). Remember to point the environment variables X509_USER_CERT and X509_USER_KEY to your Grid certificate and key, if they are not in the default area.

Configure VOMS

We can now create proxies, but not proxies with VO extensions. For that, we need to configure the VOMS. Luckily, this is really easy! Just need to download our convenience vomses file into $HOME/.voms/vomses

mkdir -p $HOME/.voms
wget http://grid-deployment.web.cern.ch/grid-deployment/dms/dmc/vomses -O $HOME/.voms/vomses
wget -O $HOME/.voms/vomses

Now try voms-proxy-init --voms <your-vo>, and you are good to go!

$ voms-proxy-init --voms dteam
Enter GRID pass phrase:
Your identity: /DC=ch/DC=cern/OU=...
Cannot find file or dir: /usr/local/etc/vomses
Creating temporary proxy ..................................... Done
Contacting  voms.hellasgrid.gr:15004 [/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr] "dteam" Done
Creating proxy ........................................... Done

Your proxy is valid until Fri Jul 31 03:34:08 2015
Error: verify failed.
Cannot verify AC signature!

$ voms-proxy-info --all
subject   : /DC=ch/DC=cern/OU=...
issuer    : /DC=ch/DC=cern/OU=...
identity  : /DC=ch/DC=cern/OU=...
type      : proxy
strength  : 1024 bits
path      : /tmp/x509up_uxxx
timeleft  : 11:59:43
key usage : Digital Signature, Key Encipherment
=== VO dteam extension information ===
VO        : dteam
subject   : /DC=ch/DC=cern/OU=...
issuer    : /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr
attribute : /dteam/Role=NULL/Capability=NULL
timeleft  : 11:59:39
uri       : voms.hellasgrid.gr:15004

Note that you can ignore the "Error: verify failed" message.

Let's verify our new proxy with the gfal2-util that we had installed!

$ source gfal2/bin/activate
(gfal2) $ gfal-ls gsiftp://dpmhead-rc.cern.ch/dpm/cern.ch/home
alice
atlas
cms
dteam
lhcb

homebrew-dmc's People

Contributors

andrea-manzi avatar ayllon avatar gbitzes avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

andrea-manzi

homebrew-dmc's Issues

[Question] gfal2-python make install failed

Hello,
I tried to install gfal2-python, but it failed when make install:

==> make install
Last 15 lines from /Users/xxx/Library/Logs/Homebrew/gfal2-python/02.make:
[ 87%] Building CXX object src/CMakeFiles/gfal-py.dir/GfaltParams.cpp.o
cd /tmp/gfal2-python-20180306-42457-1tx8vuw/src && /usr/local/Homebrew/Library/Homebrew/shims/super/clang++  -DGFAL2_PYTHON_VERSION=\"1.9.3\" -D_FILE_OFFSET_BITS=64 -Dgfal_py_EXPORTS -I/usr/local/include -I/usr/local/Cellar/pcre/8.41/include -I/usr/local/Cellar/glib/2.54.3/include/glib-2.0 -I/usr/local/Cellar/glib/2.54.3/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/gfal2/2.15.0/include/gfal2 -I/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/include/python3.6m  -O2 -fPIC   -o CMakeFiles/gfal-py.dir/gfal2.cpp.o -c /tmp/gfal2-python-20180306-42457-1tx8vuw/src/gfal2.cpp
cd /tmp/gfal2-python-20180306-42457-1tx8vuw/src && /usr/local/Homebrew/Library/Homebrew/shims/super/clang++  -DGFAL2_PYTHON_VERSION=\"1.9.3\" -D_FILE_OFFSET_BITS=64 -Dgfal_py_EXPORTS -I/usr/local/include -I/usr/local/Cellar/pcre/8.41/include -I/usr/local/Cellar/glib/2.54.3/include/glib-2.0 -I/usr/local/Cellar/glib/2.54.3/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/gfal2/2.15.0/include/gfal2 -I/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/include/python3.6m  -O2 -fPIC   -o CMakeFiles/gfal-py.dir/LoggingHelper.cpp.o -c /tmp/gfal2-python-20180306-42457-1tx8vuw/src/LoggingHelper.cpp
cd /tmp/gfal2-python-20180306-42457-1tx8vuw/src && /usr/local/Homebrew/Library/Homebrew/shims/super/clang++  -DGFAL2_PYTHON_VERSION=\"1.9.3\" -D_FILE_OFFSET_BITS=64 -Dgfal_py_EXPORTS -I/usr/local/include -I/usr/local/Cellar/pcre/8.41/include -I/usr/local/Cellar/glib/2.54.3/include/glib-2.0 -I/usr/local/Cellar/glib/2.54.3/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/gfal2/2.15.0/include/gfal2 -I/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/include/python3.6m  -O2 -fPIC   -o CMakeFiles/gfal-py.dir/GfaltParams.cpp.o -c /tmp/gfal2-python-20180306-42457-1tx8vuw/src/GfaltParams.cpp
[100%] Linking CXX shared module python2/gfal2.so
cd /tmp/gfal2-python-20180306-42457-1tx8vuw/src && /usr/local/Cellar/cmake/3.10.2/bin/cmake -E cmake_link_script CMakeFiles/gfal-py.dir/link.txt --verbose=1
/usr/local/Homebrew/Library/Homebrew/shims/super/clang++ -O2 -bundle -Wl,-headerpad_max_install_names  -o python2/gfal2.so CMakeFiles/gfal-py.dir/Directory.cpp.o CMakeFiles/gfal-py.dir/File.cpp.o CMakeFiles/gfal-py.dir/GErrorWrapper.cpp.o CMakeFiles/gfal-py.dir/Gfal2Context.cpp.o CMakeFiles/gfal-py.dir/GfaltParams.cpp.o CMakeFiles/gfal-py.dir/LoggingHelper.cpp.o CMakeFiles/gfal-py.dir/gfal2.cpp.o  -L/usr/local/Cellar/gfal2-python/1.9.3/opt/gettext/lib  -L/usr/local/Cellar/glib/2.54.3/lib  -L/usr/local/Cellar/gfal2/2.15.0/lib -Wl,-rpath,/usr/local/Cellar/gfal2-python/1.9.3/opt/gettext/lib -Wl,-rpath,/usr/local/Cellar/glib/2.54.3/lib -Wl,-rpath,/usr/local/Cellar/gfal2/2.15.0/lib /usr/local/lib/libboost_python-mt.dylib -lglib-2.0 -lintl -lgthread-2.0 -lglib-2.0 -lintl -lgfal2 -lglib-2.0 -lintl -lgfal_transfer -lgfal2 -lglib-2.0 -lintl /usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6.dylib -lgthread-2.0 -lgfal_transfer /usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6.dylib
Undefined symbols for architecture x86_64:
  "boost::python::detail::init_module(PyModuleDef&, void (*)())", referenced from:
      _PyInit_gfal2 in gfal2.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [src/python2/gfal2.so] Error 1
make[1]: *** [src/CMakeFiles/gfal-py.dir/all] Error 2
make: *** [all] Error 2

It seems will install python3 first due to gfal2-python.rb#L11, and use python3 to make install
(I am not sure it relate to this problem...)

[Question] gfal-copy failed with Segmentation fault

Hello again,
due to this question is not related to my pervious one, I create a new issue.

I need to use gfal2 to run Rucio command (download/upload),
but after running to this line leads to Segmentation fault: 11.

> /opt/rucio/lib/rucio/rse/protocols/gfal.py(369)__gfal2_copy()
-> ret = ctx.filecopy(params, str(src), str(dest))
(Pdb)
Segmentation fault: 11

So I try to initial a virtualenv (follow README of this project),
and run gfal-copy command also get error: Segmentation fault: 11.

(.venv) Mandyde-MacBook-Pro:rucio root# gfal-copy README.rst xroot://<our_storage>//eos/tmp/README.rst
180309 17:23:21 9735 secgsi_InitProxy: cannot access private key file: /var/root/.globus/userkey.pem
Segmentation fault: 11

My MacBook OS version: 10.12.6
Python version: 2.7.10

davix upgrade fails on MacOs Mojave

Hi,

davix upgrade fails from 0.6.9 to 0.7.2 on my mac:

10:56 $ brew upgrade davix
==> Upgrading 1 outdated package:
cern-fts/dmc/davix 0.6.9-1 -> 0.7.2-1
==> Upgrading cern-fts/dmc/davix
==> Cloning https://github.com/cern-fts/davix.git
Updating /Users/cecco/Library/Caches/Homebrew/davix--git
==> Checking out branch master
Already on 'master'
Your branch is up to date with 'origin/master'.
HEAD is now at c8063b7 RELEASE: 0.7.3
Entering 'deps/googletest'
/Users/cecco/Library/Caches/Homebrew/davix--git/deps/googletest
==> cmake -DENABLE_THIRD_PARTY_COPY=ON . -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/davix/0.7.2-1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_FRAM
Last 15 lines from /Users/cecco/Library/Logs/Homebrew/davix/01.cmake:
-- Found gsoap: /usr/local/lib/libgsoap.a
-- Using gsoapssl flags -DWITH_OPENSSL;-DWITH_GZIP;-DWITH_DOM;-I/usr/local/Cellar/gsoap/2.8.89/include
CMake Error at src/modules/copy/CMakeLists.txt:64 (SET_TARGET_PROPERTIES):
  SET_TARGET_PROPERTIES called with incorrect number of arguments.


-- generate pkgconfig file for davix_copy under /tmp/davix-20190827-8981-19n4jwo/src/modules/copy/davix_copy.pc
CMake Error at src/CMakeLists.txt:40 (set_target_properties):
  set_target_properties called with incorrect number of arguments.


-- generate pkgconfig file for libdavix under /tmp/davix-20190827-8981-19n4jwo/src/davix.pc
-- Configuring incomplete, errors occurred!
See also "/tmp/davix-20190827-8981-19n4jwo/CMakeFiles/CMakeOutput.log".
See also "/tmp/davix-20190827-8981-19n4jwo/CMakeFiles/CMakeError.log".

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
  https://github.com/cern-fts/homebrew-dmc/issues

cern-it-sdc-id/dmc/davix fails to build

davix fails to build under Mac OS el capitan. Build hangs after:

-- Checking for module 'gsoapssl++'
-- Found gsoapssl++, version 2.8
-- - wsdlh : /usr/local/bin/wsdl2h
-- - SOAPCPP2 : /usr/local/bin/soapcpp2

log output can be find in log.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.