Comments (3)
If you're using pip to install transformers into the MA container, that package still won't be in the Triton server container once MA starts it.
You could use a launch mode other than Docker. See here. Local, C API, and remote modes should all work, depending on your needs. Triton server will need to be accessible from within the container.
from model_analyzer.
Model Analyzer] Initializing GPUDevice handles
[Model Analyzer] Using GPU 0 NVIDIA A100-SXM4-40GB with UUID GPU-d9a0447f-f8fa-9d2f-79fc-ecf2567dacc2
[Model Analyzer] WARNING: Overriding the output model repo path "./rerenker_output1"
[Model Analyzer] Starting a local Triton Server
[Model Analyzer] Loaded checkpoint from file /model_repositories/checkpoints/0.ckpt
[Model Analyzer] GPU devices match checkpoint - skipping server metric acquisition
[Model Analyzer]
[Model Analyzer] Starting quick mode search to find optimal configs
[Model Analyzer]
[Model Analyzer] Creating model config: reranker_config_default
[Model Analyzer]
[Model Analyzer] Creating model config: bge_reranker_v2_onnx_config_default
[Model Analyzer]
[Model Analyzer] Profiling reranker_config_default: client batch size=1, concurrency=24
[Model Analyzer] Profiling bge_reranker_v2_onnx_config_default: client batch size=1, concurrency=8
[Model Analyzer]
[Model Analyzer] perf_analyzer took very long to exit, killing perf_analyzer
[Model Analyzer] perf_analyzer did not produce any output.
[Model Analyzer] Saved checkpoint to model_repositories/checkpoints/1.ckpt
[Model Analyzer] Creating model config: reranker_config_0
[Model Analyzer] Setting instance_group to [{'count': 1, 'kind': 'KIND_GPU'}]
[Model Analyzer] Setting max_batch_size to 1
[Model Analyzer] Enabling dynamic_batching
[Model Analyzer]
[Model Analyzer] Creating model config: bge_reranker_v2_onnx_config_0
[Model Analyzer] Setting instance_group to [{'count': 1, 'kind': 'KIND_GPU'}]
[Model Analyzer] Setting max_batch_size to 1
[Model Analyzer] Enabling dynamic_batching
[Model Analyzer]
[Model Analyzer] Profiling reranker_config_0: client batch size=1, concurrency=2
[Model Analyzer] Profiling bge_reranker_v2_onnx_config_0: client batch size=1, concurrency=2
[Model Analyzer]
[Model Analyzer] perf_analyzer took very long to exit, killing perf_analyzer
[Model Analyzer] perf_analyzer did not produce any output.
[Model Analyzer] No changes made to analyzer data, no checkpoint saved.
Traceback (most recent call last):
File "/opt/app_venv/bin/model-analyzer", line 8, in
sys.exit(main())
File "/opt/app_venv/lib/python3.10/site-packages/model_analyzer/entrypoint.py", line 278, in main
analyzer.profile(
File "/opt/app_venv/lib/python3.10/site-packages/model_analyzer/analyzer.py", line 124, in profile
self._profile_models()
File "/opt/app_venv/lib/python3.10/site-packages/model_analyzer/analyzer.py", line 233, in _profile_models
self._model_manager.run_models(models=models)
File "/opt/app_venv/lib/python3.10/site-packages/model_analyzer/model_manager.py", line 145, in run_models
self._stop_ma_if_no_valid_measurement_threshold_reached()
File "/opt/app_venv/lib/python3.10/site-packages/model_analyzer/model_manager.py", line 239, in _stop_ma_if_no_valid_measurement_threshold_reached
raise TritonModelAnalyzerException(
model_analyzer.model_analyzer_exceptions.TritonModelAnalyzerException: The first 2 attempts to acquire measurements have failed. Please examine the Tritonserver/PA error logs to determine what has gone wrong.
i am also getting same error any comment
from model_analyzer.
Your error is not identical. Please open a new issue for your error with full reproduction steps.
Note that we do not actively monitor closed issues or respond to questions on them, so the best route is always to open a new issue if there is not one open for the identical one. For bugs/errors, it is uncommon for issues to be identical but does happen.
from model_analyzer.
Related Issues (20)
- use of time intervals in model analyzer HOT 2
- [Bug] failed to load 'maxvit_rmlp_base', failed to poll from model repository (model_output path is the same) HOT 4
- How can i use perf_analyzer with python backend ? HOT 3
- Ensemble Model search error when specify perf_analyzer_flags.input-data HOT 3
- Model Analyzer - NO Gpu Metrics collection HOT 2
- Model Analyzer - NO Gpu Metrics collection
- Sending multiple shapes binary input data HOT 5
- Understanding GPU utilization HOT 5
- --triton-launch-mode=remote HOT 4
- custom samples HOT 1
- model analyser stuck HOT 2
- detailed report not generating HOT 2
- Time intervals in model analyzer
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 0: invalid start byte HOT 5
- How to sweep GPUs using Model Analyzer HOT 1
- How to analyze large models like llama 3 70B that requires model parallelism? HOT 3
- wrong with --triton-launch-mode=remote HOT 1
- DCGM initialization error HOT 1
- Deleted
- model_analyzer.model_analyzer_exceptions.TritonModelAnalyzerException: The first 2 attempts to acquire measurements have failed. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from model_analyzer.