nodesource-addons's People
Forkers
shatalovyaroslav activeeon-bot tobwiens paraita maelaudren marcocast pletessier vinseon jrochas gheon mykhailenko tao-xinxiu mcompastie luispinedamx fviale nebilbenmabrouk mbenguig alexfalessi mklkun aminelouati oanabiancaschiopunodesource-addons's Issues
Improve description of infrastructures parameters
For the optional infrastructures parameters which have a default value, it should be presented in the description.
All cloud infrastructures should have in their form a field to upload the user's credentials
Currently, the rm.cred that is used by the node to register to the RM is located in the node.jar. If ever the credentials change on the server side, then it will never be possible for the node to connect. To prevent this problem, the best way is to ask the user to provide its own credentials, so we need to add this in the forms of all cloud infrastructures, like for the LocalInfrastructure.
Azure Infrastructure not deploying
Description
-
Azure nodes don't get deployed. When using the right credentials, the logs show a successful deployment, but the node is not even shown as "deploying".
-
The VM, however, gets created and is accessible through ssh.
-
Another issue which might be consequence to this one, is that the VM is not deleted upon undeployment. The VM and all its related resources are left there, incurring costs.
-
Upon accessing the deployed VM we can see that
node.jar
is correctly downloaded, but the launchingnohup
command fails asjava
is not installed in a generic machine:
root@azuretest:/var/lib/waagent/custom-script/download/0# cat stderr 2019-05-22 13:34:24 URL:http://40.89.153.167:8080/rest/node.jar [132930488/132930488] -> "node.jar" [1] nohup: failed to run command 'java': No such file or directory
root@azuretest:/var/lib/waagent/custom-script/download/0# cat stdout script-ID 3fa86921-4cc4-4457-aa4c-776ff5e93bf9
To investigate/fix:
- Can we deploy a node with a VM image with java pre-installed?
- Verify that the protocol (PNP, PAMR) and URL parameters are correct.
- Add java installation to the launching script, to avoid using a custom image.
Scheduler.log:
o.o.p.r.c.RMCore] Node source azuretest has been successfully edited o.o.p.r.c.RMCore] Deploy node source azuretest requested by admin o.p.r.n.i.AzureInfrastructure] Validating parameters : [HIDDEN] o.o.p.r.n.NodeSource] [azuretest] Activating the policy Static Policy user access type [ALL], provider access type [ME] o.o.p.r.n.p.NodeSourcePolicy] [azuretest] Acquiring all nodes .r.n.i.ConnectorIaasController] Creating Azure infrastructure : {"credentials":HIDDEN,"id":"azuretest","type":"azure","toBeRemovedOnShutdown":false} .r.n.i.ConnectorIaasController] Azure infrastructure created .m.a.a.AuthenticationAuthority] [Correlation ID: XXXX-XXXX-XXXX] Instance discovery was successful .r.n.i.ConnectorIaasController] Total existing Instances By Infrastructure Id : 0 .r.n.i.ConnectorIaasController] InstanceJson : {"image":"proactive-scaleset","number":"1","credentials":HIDDEN .r.n.i.ConnectorIaasController] Instances ids created : [62038d76-e67a-4951-879c-f64e3b3f98a4] .o.p.r.n.i.AzureInfrastructure] Instances ids created or retrieved : [62038d76-e67a-4951-879c-f64e3b3f98a4] .r.n.i.ConnectorIaasController] Trying to execute script for instance id:62038d76-e67a-4951-879c-f64e3b3f98a4 .r.n.i.ConnectorIaasController] Executed successfully script for instance id:62038d76-e67a-4951-879c-f64e3b3f98a4 .r.n.i.ConnectorIaasController] InstanceScriptJson: {"scripts":["/bin/bash -c 'echo script-ID; echo 3fa86921-4cc4-4457-aa4c-776ff5e93bf9; wget -nv http://40.89.153.167:8080/rest/node.jar; nohup java -jar node.jar -Dproactive.pamr.router.address=40.89.153.167 -DinstanceId=62038d76-e67a-4951-879c-f64e3b3f98a4 -Dproactive.useIPaddress=true -r pnp://server-test:64738/ -s azuretest -w 1 &'"]} .r.n.i.ConnectorIaasController] Script result: [{"instanceId":"62038d76-e67a-4951-879c-f64e3b3f98a4","output":"","error":""}] o.o.p.r.n.NodeSource] [azuretest] Pinging alive nodes : 0 o.o.p.r.c.RMCore] Node source azuretest has been successfully deployed
JMX RMI nodes service address uses the private IP when deployed in AWS
Inject script on boot (if available) for multiple instances
The issue is about script injection on boot and multiple instance creatio and is partially related to the issue ow2-proactive/connector-iaas#47.
On Azure & OpenStack cloud, the acquireNode
script can be injected on VM's boot.
In addition, when we have multiple instances to start, some optimizations are made inside connector-iaas' createInstance
method to start multiple VMs/instances as fast as possible using SDK features.
The problem is that the scripts need to be customized with instanceTag/id and (according to the model issue of the connector-iaas
project) this will force instances to start sequentially.
We thus need to find a way to customize the script for each instance to take advantage of clouds SDK's parallel VMs deployment features.
Code improvements
Some code improvements need to be done in the entire project according to the reviews on PR #31:
- better parameters management (using a HashMap instead of Object[]?)
- code refactoring (can be partially based on the most recent Microsoft Azure addon)
- clean unit tests
Azure: enable PNP communications
In Azure, by default, there is no domain name associated with the public IP address attached to the VMs we deployed. As a consequence, a VM only has the knowledge of its private IP address even if there is a public IP attached to it (along with open ports in the security group), so PNP will only listen on the local/private IP address which is unreachable from an external scheduler.
The problem can be solved either by generating and adding a domain name to the instance (which can be done programmatically using the SDK), or by extending the injection script to retrieve the public IP of the VM and add it as an option to the node.jar execution.
Form submission improvement
Add notifications with appropriate error names during form submission, there is currently no feedback on what is happening behind (infrastructure registration, instances creation and script injection).
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.