Comments (17)
Hello @jeffreyalien ,
I assume the pthread
error messages came from the CMakeError.log
file - however, this is an artifact of the way cmake
work, and not the direct culprit.
May I ask you to delete the build-cachelib
directory and rebuild it with ./contrib/build-package.sh -O -v cachelib 2>&1 build.log
- then (assuming it still fails), share with us the build.log
, CMakeError.log
and CMakeOutput.log
?
Thanks!
from cachelib.
@agordon ,
There is no -O option on our version of contrib/build-package.sh. We are building with the latest code from the master branch. Is there a different branch/tag that we should be using?
Here's the list of options we have available for the build-package.sh:
./contrib/build-package.sh -h
CacheLib dependencies builder
usage: build-package.sh [-BdhijStv] NAME
options:
-B skip build step
(default is to build with cmake & make)
-d build with DEBUG configuration
(default is RELEASE with debug information)
-h This help screen
-i install after build using 'sudo make install'
(default is to build but not install)
-j build using all available CPUs ('make -j')
(default is to use single CPU)
-S skip git-clone/git-pull step
(default is to get the latest source)
-t build tests
(default is to skip tests if supported by the package)
-v verbose build
from cachelib.
Apologies, my bad. No need for -O
- just remove it (it belongs to the more general build.sh
script).
from cachelib.
@agordon
Here's the build output and log files with the -v option.
build-fail-070721.zip
from cachelib.
So, here is what I think is the issue. The FindSodium.cmake is being double included. There are two version of it. One in fizz and one in cachelib. Putting the right protections as the fizz version of FindSodium does gets rid of the "Configuring Incomplete!" cmake error.
Found that by running cmake in trace more.. log attached.
cachelib_cmake.log
from cachelib.
Hello @dreddy,
Indeed, that is one of the issues (another is that the datatype
library was not using suitable parameters which caused the pthread
linking error). I just sent an email to @jeffreyalien with updated patch - I'm happy to forward it to you (shall I use your "@intel.com" email from GitHub) ?
from cachelib.
Yes agordon indeed. Kindly forward it to me.
Please use my @intel.com e-mail.
Hello @dreddy,
Indeed, that is one of the issues (another is that thedatatype
library was not using suitable parameters which caused thepthread
linking error). I just sent an email to @jeffreyalien with updated patch - I'm happy to forward it to you (shall I use your "@intel.com" email from GitHub) ?
from cachelib.
@agordon
I pulled the latest version of the CacheLib code and applied you patch. Now we're seeing this failure in attached build log. Could you check it out and let me if you need more info or how to resolve. Thanks.
build_error_during_glog.log
from cachelib.
@agordon
With your patch applied, it's getting a build error during the cachelib build now. Attached the build log.
build_error_during_cachelib_build.log
from cachelib.
@dreddy @jeffreyalien we just pushed to the repository the mentioned patch plus few other fixes - mind giving the latest git-version a try and tell if it builds better?
from cachelib.
@agordon We were finally able to build and install Cachelib with this update. We'll be doing some testing with cachebench now. I'll update you on how that goes when we're done.
from cachelib.
@agordon
We are still hitting issues; now when trying to execute cachelib. Here's a summary of the problem:
After manually cloning the following dependancies folly, wangle, fbthrift, fizz to CacheLib/cachelib/external. I am able to successfully build CacheLib from the latest CacheLib git hub code. However when attempting to run cachebench I recieve the following error.
CacheLib/build-cachelib/cachebench/cachebench: error while loading shared libraries: libthriftcpp2.so.1.0.0: cannot open shared object file: No such file or directory
I am unfamiliar with cmake files but I did attempt to add the option to build shared libraries for fbthrift. Adding BUILD_SHARED_LIBS=ON to CMakeCache.txt and regenerating the make file and then rebuilding fbthrift did not resolve the issue.
from cachelib.
Hello @jeffreyalien ,
After manually cloning the following dependancies folly, wangle, fbthrift, fizz to CacheLib/cachelib/external. I am able to successfully build CacheLib from the latest CacheLib git hub code.
Question: why do you need to manually git-clone the dependencies ? the script ./contrib/build.sh
does it automatically.
However when attempting to run cachebench I recieve the following error.
CacheLib/build-cachelib/cachebench/cachebench: error while loading shared libraries: libthriftcpp2.so.1.0.0: cannot open shared object file: No such file or directory
Please check if you have an opt/cachelib
subdirectory - the new build scripts install all dependencies AND the cachebench
binary there.
When cachelib
is build using the ./contrib/build.sh
, it automatically configures the binary to look for the libraries in the ../lib
and ../lib64
relative directories (using a linker feature called RPATH
).
So if you run ./opt/cachelib/bin/cachebench
, it should automatically find all the required libraries in ./opt/cachelib/lib/
.
I am unfamiliar with cmake files but I did attempt to add the option to build shared libraries for fbthrift. Adding BUILD_SHARED_LIBS=ON to CMakeCache.txt and regenerating the make file and then rebuilding fbthrift did not resolve the issue.
That is correct (the BUILD_SHARED_LIBS=ON) - and new build script builds all dependencies with this flag, and all the shared libraries should be installed in the subdirectories ./opt/cachelib/lib
and/or ./opt/cachelib/lib64
.
Please let me know if you see these files, and if using them work for you.
from cachelib.
@agordon
Here's an explanation to your comments above.
-
Why manually git-clone dependencies?
See the attached log (build_error_during_folly.log) Without manually cloning the dependencies I get “fatal: not a git repository (or any of the parent directories): .git” when running the build.sh script. So I am interpreting that as the script does attempt to clone but fails to do so. -
Check for the opt/cachelib subdirectory
That subdirectory does exist. I copied the cachebench from there and ran it from a different directory where our automation looks for it by default. Running ./opt/cachelib/bin/cachebench it does appear that all of the dependencies are located. Now the model number is not recognized as a WDC device. As seen in Cachebench_run_error.log
build_error_during_folly.log
Cachebench_run_error.log
from cachelib.
Now the model number is not recognized as a WDC device. As seen in Cachebench_run_error.log
@jeffreyalien you can now apply your earlier patch and verify if the device is appropriately handled. Once you verify, please send out a PR to merge.
Can you confirm that your build issues are fixed so that I can close out this issue ?
from cachelib.
@agordon @sathyaphoenix
I've fixed the model number check but we're hitting this exception when reading "WriteyBytes": Exception fetching nand writes for nvme1n1. I see how to specify the field number and factor in the getBytesWritten function. But have a question on how that function knows which line in vs smart log data to read. Here's what our data looks like. Is "Physical media units written" ok?
[root@fb-yv2-s3-n4 CacheLib]# nvme wdc vs-smart-add-log /dev/nvme1n1
NVMe Status:SUCCESS: The command completed successfully(0)
SMART Cloud Attributes :-
Physical media units written - 0 2068679535362048
Physical media units read - 0 2658736152236032
Bad user nand blocks - Raw 0
Bad user nand blocks - Normalized 100
Bad system nand blocks - Raw 0
from cachelib.
But have a question on how that function knows which line in vs smart log data to read. Here's what our data looks like. Is "Physical media units written" ok?
You have to write the logic to do that. You can use the function getBytesWritten by passing the right arguments. See here for some examples here https://github.com/facebookincubator/CacheLib/blob/master/cachelib/cachebench/util/NandWrites.cpp#L154
I'll close this issue out since the build issue is resolved.
from cachelib.
Related Issues (20)
- Fail to build dependency fbthrift (with errors reported in fmt) HOT 5
- make clean option for contrib/build.sh HOT 1
- build error about fizz on ubuntu22.04 HOT 2
- CDN trace expected behavior HOT 2
- Enable FDP for CacheBench HOT 26
- qDepth Support for NVM Cache HOT 6
- Questions about trace files when running cachebench HOT 2
- Running simple-cache-example gives an error, flag 'v' was defined more than once HOT 6
- OSS build broken as of May, 2024 -> PRs are all blocked HOT 7
- No build support for Fedora37 OS HOT 4
- failed to build CacheLib following document HOT 2
- Build fails on debian-10 HOT 2
- Segmentation fault while fetching refcount HOT 2
- Minimum Limit For Cache Allocation? HOT 1
- build failed when building dependency 'fbthrift' HOT 3
- Build issue with CacheLib with missing source files HOT 3
- [Seeking Volunteers] Add new builds to CacheLib HOT 2
- Build is failing with error: ‘fmt::v10::detail::type_is_unformattable_for<const facebook::cachelib::navy::Status, char> _’ has incomplete type 1600 | type_is_unformattable_for<T, typename Context::char_type> _; | ^ HOT 4
- NavySetup should not involve the MockDevice
- How to configure folly for logging CacheLib
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cachelib.