controlm / automation-api-community-solutions Goto Github PK
View Code? Open in Web Editor NEWControl-M Automation API community solutions
License: Other
Control-M Automation API community solutions
License: Other
I tried to follow the tutorial here using the workbench appliance running in VirtualBox and the docker build
step fails with a connection refused being returned by the workbench server.
--2019-10-07 18:48:22-- https://localhost:8443/automation-api/ctm-cli.tgz
Resolving localhost (localhost)... 127.0.0.1, ::1
Connecting to localhost (localhost)|127.0.0.1|:8443... failed: Connection refused.
Connecting to localhost (localhost)|::1|:8443... failed: Cannot assign requested address.
MacOS: 10.14.6
Docker: 19.03.2
VirtualBox: 6.0.12
Workbench Image: 9.0.18.300-20190218.133426-1
Hi,
I am following the example kubernetes-statefulset-agent-using-pvc to run ctm on kuberntes. I am able to build the image but when I execute it I get java.lang.OutOfMemoryError. I believe I fix it increasing the heap size. Does someone know where can I set it up?
Here are the logs:
│ info: Making SSL trust all certificates and all hostnames │
│ info: setting server to agent port: 7006 │
│ info: setting agent to server port: 7005 │
│ info: setting agent name (alias): statefulset-agent-0 │
│ info: setting primary Control-M Server: 10.228.169.20 │
│ info: setting authorized Control-M Server host │
│ info: setting agent communication type to persistent │
│ error: Exception in thread "Exec Stream Pumper" java.lang.OutOfMemoryError: Java heap space │
│ at java.base/java.util.Arrays.copyOf(Arrays.java:3745) │
│ at java.base/java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120) │
│ at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95) │
│ at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:156) │
│ error: │
│ at org.apache.commons.exec.StreamPumper.run(StreamPumper.java:108) │
│ at java.base/java.lang.Thread.run(Thread.java:834) │
Hi,
I am trying to build the control m agent image using the instructions provided on https://github.com/controlm/automation-api-community-solutions/tree/master/3-infrastructure-as-code-examples/kubernetes-statefulset-agent-using-pvc. Our control M EM version is 9.20.0
I'm getting the following error while building the image
=> ERROR [builder 10/14] RUN ctm provision image Agent.Linux && echo installation ended successfully 17.1s
------
> [builder 10/14] RUN ctm provision image Agent.Linux && echo installation ended successfully:
#14 0.567 debug: Locating java command
#14 0.671 info: Located java at:/bin/java
#14 0.671 info: downloading https://10.228.158.10:8446/automation-api/utils/control-m.services.provision-9.20.0.jar into /home/controlm/.ctm/control-m.services.provision-9.20.0.jar
info: 4MB/4MB precent: 100%
#14 5.888 debug: starting command: /bin/java -jar /home/controlm/.ctm/control-m.services.provision-9.20.0.jar -image "Agent.Linux" -server https://10.228.158.10:8446/automation-api -action image -environment myenv -ctms "" -name "" -port "" -cert 0 -file ""
#14 6.148 info: Making SSL trust all certificates and all hostnames
#14 8.382 info: image descriptor Agent.Linux was successfully downloaded and parsed
#14 8.383 info: handling product DRKAI.9.0.00_Linux-x86_64.tar.Z
#14 8.559 info: downloading https://10.228.158.10:8446/automation-api/silent/agent.linux.xml to /home/controlm/.ctmTemp/agent.linux.xml
#14 10.03 info: downloading https://10.228.158.10:8446/automation-api/descriptors/ProvisionConfig.json to /home/controlm/.ctmTemp/ProvisionConfig.json
#14 10.06 info: downloading https://10.228.158.10:8446/automation-api/DRKAI.9.0.00_Linux-x86_64.tar.Z to /home/controlm/.ctmTemp/DRKAI.9.0.00_Linux-x86_64.tar.Z
#14 10.08 info: file https://10.228.158.10:8446/automation-api/DRKAI.9.0.00_Linux-x86_64.tar.Z not found, may be valid in some cases...
#14 10.08 info: 100% (0MB/0MB) - downloading ended
#14 17.06 info: Failed to deploy new image
#14 17.06 error: java.io.IOException: Product 'DRKAI.9.0.00_Linux-x86_64.tar.Z' listed in image 'Agent.Linux' not found
#14 17.06
#14 17.06 info: exiting provision process with exit code (0-ok):6
#14 17.09 debug: image failed: exit code: 6 for '/bin/java -jar /home/controlm/.ctm/control-m.services.provision-9.20.0.jar -image "Agent.Linux" -server https://10.228.158.10:8446/automation-api -action image -environment myenv -ctms "" -name "" -port "" -cert 0 -file ""'
I tried doing curl on https://10.228.158.10:8446/automation-api/DRKAI.9.0.00_Linux-x86_64.tar.Z but got the following error response
{ "errors" : [
{"message":"Not Found", "code" : "404", "uri" : "/DRKAI.9.0.00_Linux-x86_64.tar.Z"} ]
}
I also tried for the agent image Agent_18.Linux but got the same response.
Could you please help me to resolve this issue?
Thanks
Anshul
The Control M Agent 9.0.21.000+ require a minimum Java version of 11.0.12+.
I have tried to replace openjdk package with Amazon Corretto Headed/Headless package and getting issues with using the automation api to provision the Agent 9.0.21.000 image.
RUN yum install -y java-11-amazon-corretto-headless
changing the package java-1.8.0-openjdk to java-11-amazon-corretto-headless or java-11-amazon-corretto results in the error below.
Cannot find Java. The Java major version must be 11. The minor version must be 11.0.12 or higher. Set the BMC_INST_JAVA_HOME environment variable to the Java home path and then rerun the installation. Refer to Control-M Documentation for more information.
�[32minfo�[39m: Failed to deploy new image
�[91m�[31merror�[39m: org.apache.commons.exec.ExecuteException: return error for product DRFZ4.9.0.21.000_Linux-x86_64.tar.z-DEPLOY: using command /home/tomcat/.ctmTemp/DRFZ4.9.0.21.000_Linux-x86_64.tar/setup.sh -silent /home/tomcat/.ctmTemp/agent.linux.xml. rc=1 (Exit value: 1)
�[0m�[32minfo�[39m: exiting provision process with exit code (0-ok):6
�[91m�[31merror�[39m: provision image failed
�[0mThe command '/bin/sh -c cd && ctm provision image $AGENT_DEPLOY_FILE' returned a non-zero code: 1
Do you have a solution for this, or a workaround?
I'm setting up something similar to what you've got in your jenkins CICD integration ( automation-api-community-solutions/2-cicd-tooling-integration/integrating-Control-M-into-DevOps-lifecycle/) but am struggling to understand how this approach will work for a project with multiple folders of jobs... would it be normal to need to create a new jenkins pipeline for each folder as I would think there could be hundreds of these in each project or would the script need to be adapted to cat all the jobs into one master file just for validating build
The lcps_to_ccps.py script converts local connection profiles to centralized connection profiles which includes changing the value of the Password field in the exported JSON to a secret. I found that when trying to convert Application Integrator profiles the script does not account for the fact that the password field name for AI jobs is named AI-Password instead.
To illustrate this below is a profile I exported using CTM CLI:
"LOCKBOX": {
"Type": "ConnectionProfile:ApplicationIntegrator:ZCONNECTWF",
"AI-Timeout (minutes)": "60",
"AI-Status Interval (seconds)": "600",
"AI-Password": "*****",
"AI-Zuora Workflow ID": "82290",
"AI-Host": "workflow.apps.zuora.com",
"TargetAgent": "agentname",
"TargetCTM": "servername"
}
When I use the script to generate the temporary JSON file for that same profile you can see that it has added the Password element:
"LOCKBOX": {
"Type": "ConnectionProfile:ApplicationIntegrator:ZCONNECTWF",
"AI-Timeout (minutes)": "60",
"AI-Status Interval (seconds)": "600",
"AI-Password": "*****",
"AI-Zuora Workflow ID": "82290",
"AI-Host": "workflow.apps.zuora.com",
"Centralized": true,
"Password": {
"Secret": "lockbox_secret"
}
For the moment I am going to use the workaround of modifying the JSON file before telling the script to deploy it but I am sure the script could be refactored to handle AI passwords properly. When I have time I will likely create my own fork and work on that.
Hi,
I am following the tutorial kubernetes-statefulset-agent-using-pvc to run control m agents on Kubernetes and have followed all the steps as mentioned in the read me.
I am successfully able to run the command kubectl exec -it statefulset-agent-0 -- tcsh -c ag_ping
but when I try ctmping -HOSTID statefulset-agent-0 on the server it fails with the error unknown host as the statefulset-agent- doesn't exist
Is there same sort of configuration (networking) that is required?
As we are using the ConnectionInitiation=AgentToSever does it mean that everytime server and agent communication needs to happen agent will make a connection and hence server doesn't need to know about the the location of agent as long as agent is able to communicate with server?
Hi,
I'm working on the installation of control-m agent on a docker container and using the pattern where control-m agent has kubernetes service object attached to it which provides a static IP to the agent for incoming connections from the server.
Please note, for the installation of control-m agent on the docker image, I am executing the steps that we will perform on any Linux VM i.e. downloading the DRKAI package, unzipping and running setup.sh command. I didn't use ctm cli as it requires the EM endpoint and credentials at the build time which means that if you have got multiple environments each having its own EM then you need to build different image for each of them and that I believe is an anti-pattern in container world.
Also, I am running the setup.sh at the build time with a placeholder for server name in silent config file. Later during start up time, I replace that placeholder in ctm/data/CONFIG.dat with the actual server name. (I still need to handle authorised servers in case of HA).
Following is the snippet of my docker file
FROM centos:7
ARG CTM_HOME=/home/controlm
ARG AGENT_BINARY_URL=
ENV CTM_DIR=$CTM_HOME
RUN yum -y update \
&& yum -y install wget \
&& yum -y install sudo \
&& yum -y install net-tools \
&& yum -y install bc
# Add controlm user and root to soduers list
RUN useradd -d $CTM_DIR -m controlm \
&& echo 'root ALL=(ALL) ALL' >> /etc/sudoers \
&& echo 'controlm ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
USER controlm
WORKDIR $CTM_DIR
RUN wget -q $AGENT_BINARY_URL
RUN tar -xvf $ARTIFACT_NAME
RUN rm -rf $ARTIFACT_NAME \
&& chmod 755 setup.sh \
&& chmod -R 755 Setup_files
COPY install-controlm-silent-config.xml install-controlm-silent-config.xml
RUN ./setup.sh -silent install-controlm-silent-config.xml
RUN rm -rf Setup_files
ADD run_ctm.sh run_ctm.sh
RUN sudo chmod 755 run_ctm.sh
RUN echo "source $CTM_DIR/.bash_profile" >> $CTM_DIR/.bashrc
EXPOSE 7000-8000
CMD $CTM_DIR/run_ctm.sh
run.sh (start up script)
function run_ctrlm {
if [ ! $CTM_SERVER_HOST ] ; then
echo "Environment variable CTM_SERVER_HOST is missing."
exit 1
fi
source ~/.bash_profile
sed -i -e "s/CTM_SERVER_HOST/$CTM_SERVER_HOST/g" $CTM_DIR/ctm/data/CONFIG.dat
sed -i -e 's|/bin/su - $agent_owner -c "||g' -e 's|ALL"|ALL|g' $CTM_DIR/ctm/scripts/rc.agent_user
$CTM_DIR/ctm/scripts/rc.agent_user start
}
run_ctrlm
while /bin/true; do
sleep 60
done
install-controlm-silent-config.xml
<AutomatedInstallation langpack="eng">
<target.product>Control-M/Agent 9.0.20.000</target.product>
<agent.parameters>
<entry key="field.Authorized.Controlm.Server.Host" value="CTM_SERVER_HOST"/>
<entry key="ctm_agent.Tracker.Event.Port" value="7035"/>
<entry key="field.Agent.To.Server.Port.Number" value="7005"/>
<entry key="ctm_agent.Tcp_ip.Timeout" value="60"/>
<entry key="field.Server.To.Agent.Port.Number" value="7006"/>
<entry key="Ignore.Disabling.Agent.Failure" value="false"/>
<entry key="ctm_agent.Force.Upgrade" value="false"/>
<entry key="field.Primary.Controlm.Server.Host" value="CTM_SERVER_HOST"/>
</agent.parameters>
</AutomatedInstallation>
Can you please provide the feedback on this approach?
Also, I want to use persistent volume claim to persist the logs and output for troubleshooting if the pod goes down. So far I understand that I should definitely persist proclog and output directory, but are there some other folders we should persist?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.