This project forked from ghoneycutt/puppet-scm
Puppet module to deploy Cloudera Manager and Cloudera's Distribution, including Apache Hadoop (CDH).
Home Page: http://forge.puppetlabs.com/razorsedge/cloudera
License: Other
I try a very simple setup :
# ClouderaManager node
node /^cm\d+.vagrant.dev$/ {
class { 'cloudera':
cm_server_host => $::hostname,
install_cmserver => true,
}
}And the server fail with an exception :
[vagrant@cm1 ~]$ sudo /etc/init.d/cloudera-scm-server-db status
could not change directory to "/home/vagrant"
pg_ctl: server is running (PID: 11677)
/usr/bin/postgres "-D" "/var/lib/cloudera-scm-server-db/data"
[vagrant@cm1 ~]$ sudo /etc/init.d/cloudera-scm-server status
cloudera-scm-server dead but pid file exists
[vagrant@cm1 ~]$ sudo cat /var/log/cloudera-scm-server/cloudera-scm-server.out
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.cloudera.server.cmf.TrialState': Cannot resolve reference to bean 'entityManagerFactoryBean' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is javax.persistence.PersistenceException: [PersistenceUnit: cmf.server] Unable to build EntityManagerFactory
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:616)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:148)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1003)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:907)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
at com.cloudera.server.cmf.Main.bootstrapSpringContext(Main.java:344)
at com.cloudera.server.cmf.Main.<init>(Main.java:220)
at com.cloudera.server.cmf.Main.main(Main.java:194)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is javax.persistence.PersistenceException: [PersistenceUnit: cmf.server] Unable to build EntityManagerFactory
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:149)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:102)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1440)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:247)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
... 17 more
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: cmf.server] Unable to build EntityManagerFactory
at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:924)
at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:899)
at org.hibernate.ejb.HibernatePersistence.createEntityManagerFactory(HibernatePersistence.java:59)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:63)
at com.cloudera.enterprise.CommonMain.setupHibernate(CommonMain.java:171)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:107)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:60)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:142)
... 22 more
Caused by: org.hibernate.service.classloading.spi.ClassLoadingException: HHH010003: JDBC Driver class not found: com.mysql.jdbc.Driver
at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.configure(C3P0ConnectionProvider.java:142)
at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:75)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:159)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:131)
at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.buildJdbcConnectionAccess(JdbcServicesImpl.java:223)
at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:89)
at org.hibernate.service.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:75)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:159)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:131)
at org.hibernate.cfg.Configuration.buildTypeRegistrations(Configuration.java:1797)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1755)
at org.hibernate.ejb.EntityManagerFactoryImpl.<init>(EntityManagerFactoryImpl.java:96)
at org.hibernate.ejb.Ejb3Configuration.buildEntityManagerFactory(Ejb3Configuration.java:914)
... 29 more
Caused by: org.hibernate.service.classloading.spi.ClassLoadingException: Unable to load class [com.mysql.jdbc.Driver]
at org.hibernate.service.classloading.internal.ClassLoaderServiceImpl.classForName(ClassLoaderServiceImpl.java:149)
at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.configure(C3P0ConnectionProvider.java:139)
... 41 more
Caused by: java.lang.ClassNotFoundException: Could not load requested class : com.mysql.jdbc.Driver
at org.hibernate.service.classloading.internal.ClassLoaderServiceImpl$AggregatedClassLoader.findClass(ClassLoaderServiceImpl.java:296)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.hibernate.service.classloading.internal.ClassLoaderServiceImpl.classForName(ClassLoaderServiceImpl.java:146)
... 42 more
I do not understand why do it try to use the mysql jdbc driver when the embedded db is Postgesql...
fyi :
[vagrant@cm1 ~]$ sudo yum list cloudera\*
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: mirrors.ircam.fr
* extras: mirrors.ircam.fr
* updates: miroir.univ-paris13.fr
Installed Packages
cloudera-manager-agent.x86_64 5.4.0-1.cm540.p0.165.el6 @cloudera-manager
cloudera-manager-daemons.x86_64 5.4.0-1.cm540.p0.165.el6 @cloudera-manager
cloudera-manager-server.x86_64 5.4.0-1.cm540.p0.165.el6 @cloudera-manager
cloudera-manager-server-db-2.x86_64 5.4.0-1.cm540.p0.165.el6 @cloudera-manager
Thanks
I have two RedHat 6.7 servers. One is a Puppet Master, and one is a Puppet Agent. On the Puppet Master I installed two dependencies of the Cloudera module and the Cloudera Module itself. I used this link (and the links to its dependencies): https://forge.puppet.com/razorsedge/cloudera
I do not want to deviate from the standard module whenever possible. I was having errors so I commented out offending stanzas from the different manifests that came with the module. There were many "duplicate declaration" errors. I do not know if commenting out those stanzas will later present a problem. But it seems to have helped.
When I run the "puppet agent -t -d" command on the Puppet Agent, I get this error:
Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error; Error while evaluating a Resource Statement, Evaluation Error: Illegal Resource Type expression, expected result to be a type name, or untitled Resource, got Type[Class] at /etc/puppetlabs/code/environments/production /modules/cloudera/manifests/cdh.pp:64:3 on node FQDNofPuppetAgent
Here are the relevant lines of cdh.pp (on the Puppet Master):
64 Class {
65 require => Anchor['cloudera::cdh::begin'],
66 require => Anchro['cloudera::cdh::end'],
67 }
I tried giving the Class a name before and after the opening brace. Neither of those things worked. I tried changing "Class" to "class". But that did not work. How do I get around this error? I want the puppet manifest to apply. I think the "puppet agent -t -d" should work.
i'm using cloudera-manager-server-db-2-5.4.5-1.cm545.p0.5.el6.x86_64.rpm
couldnt find any files in var/lib/cloudera-scm-server-db/data . How to get rectify this issue.
please help me
I'm using the following configuration in site.pp:
class { '::cloudera':
cm_server_host => 'cdh-server.test.com',
install_cmserver => true,
}
I have Java Oracle JDK 8 on my Ubuntu 14.04 machine.
Later, on puppet agent I get this error:
Error: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install jdk' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package jdk
Error: /Stage[main]/Cloudera::Java/Package[jdk]/ensure: change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install jdk' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package jdk
Notice: /Stage[main]/Cloudera::Java/Exec[java-alternatives]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera::Java/Exec[java-alternatives]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm/Package[cloudera-manager-agent]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera::Cm/Package[cloudera-manager-agent]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm/Package[cloudera-manager-daemons]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera::Cm/Package[cloudera-manager-daemons]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm/File[scm-config.ini]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera::Cm/File[scm-config.ini]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm/Service[cloudera-scm-agent]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera::Cm/Service[cloudera-scm-agent]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera/Anchor[cloudera::end]: Dependency Package[jdk] has failures: true
Warning: /Stage[main]/Cloudera/Anchor[cloudera::end]: Skipping because of failed dependencies
Please let me know if I missing something.
Thanks
On a green-field installation packages are installed before the repos are created. There is a missing dependency.
Info: Applying configuration version '1384290998'
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server' returned 1: Error: Nothing to do
Error: /Stage[main]/Cloudera::Cm::Server/Package[cloudera-manager-server]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server' returned 1: Error: Nothing to do
Notice: /Stage[main]/Cloudera::Cm::Server/File[/etc/cloudera-scm-server/db.properties]: Dependency Package[cloudera-manager-server] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/File[/etc/cloudera-scm-server/db.properties]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server-db' returned 1: Error: Nothing to do
Error: /Stage[main]/Cloudera::Cm::Server/Package[cloudera-manager-server-db]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server-db' returned 1: Error: Nothing to do
Notice: /Stage[main]/Cloudera::Cm::Server/Exec[cloudera-manager-server-db]: Dependency Package[cloudera-manager-server-db] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Exec[cloudera-manager-server-db]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server-db]: Dependency Package[cloudera-manager-server-db] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server-db]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Dependency Package[cloudera-manager-server-db] has failures: true
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Dependency Package[cloudera-manager-server] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Skipping because of failed dependencies
Info: create new repo cloudera-manager in file /etc/yum.repos.d/cloudera-manager.repo
Hi! Thanks for your module!
node 'vm-cluster-nolde1.localdomain' { #inherits default {
class { 'cloudera':
cm_version => '4.8.0',
cm_server_host => 'vm-cluster-node1.localdomain',
use_parcels => true,
} ->
class { 'cloudera::cm::server': }
}The composed base url is:
Notice: /Stage[main]/Cloudera::Cm::Repo/Yumrepo[cloudera-manager]/baseurl: baseurl changed '' to 'http://archive.cloudera.com/cm4/redhat//x86_64/cm/4.8.0/'
version "6" is missing. What do I do wrong?
One of the modules this puppet-module depends on (I performed a manual search, couldn't find which one) makes use of ripienaar/concat. This conflicts with puppetlabs/concat in my environment which a lot of other modules use.
I did a bit of research and apparently this is the old name of puppetlabs/concat (joshbeard/puppet-display#4). Is there a way to resolve the conflict by specifying it somewhere in this module?
I have two RedHat 6.7 servers. One is a Puppet Master, and one is a Puppet Agent.
On the Puppet Master I installed two dependencies of the Cloudera module and the Cloudera Module itself. I used this link (and the links to its dependencies): https://forge.puppet.com/razorsedge/cloudera
I do not want to deviate from the standard module whenever possible.
When I run the "puppet agent -t -d" command on the Puppet Agent, I get this error:
Error: Could not retrieve the catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Class[Cloudera::Java5] is already declared; cannot redeclare at /etc/puppetlabs/code.../init.pp
I don't know why I received this. I am using a plain server without customizing the Cloudera module.
To work around this problem, I commented out the class of "cloudera::java5" from the init.pp file that comes with the Cloudera module on the Puppet Master server. I then ran "puppet agent -t -d" again from the Puppet Agent server.
This time I get a very similar error except it deals with a duplicate declaration of Class[Cloudera::Cm5].
What am I supposed to do to get the Cloudera module to work? Comment out many different sections of the init.pp file?
puppet about it (java is installed before):
...
class { 'cloudera':
install_java => false,
ensure => 'present',
cm_server_host => 'mainhead.es.fr.alten.com',
use_parcels => false,
}
->
class { 'cloudera::cm::server':
db_type => 'postgresql',
db_user => 'root',
db_pass => 'scm',
}
->
class { 'cloudera::cdh::hue': }
->
class { 'cloudera::cdh::mahout': }
->
class { 'cloudera::cdh::sqoop': }
->
class { 'cloudera::gplextras': }
...
Errors:
...
Notice: /Stage[main]/Cloudera::Cm::Server/Exec[scm_prepare_database]/returns: at com.cloudera.enterprise.dbutil.DbProvisioner.doMain(DbProvisioner.java:95)
Notice: /Stage[main]/Cloudera::Cm::Server/Exec[scm_prepare_database]/returns: at com.cloudera.enterprise.dbutil.DbProvisioner.main(DbProvisioner.java:110)
Notice: /Stage[main]/Cloudera::Cm::Server/Exec[scm_prepare_database]/returns: --> Error 1, giving up (use --force if you wish to ignore the error)
Error: /usr/share/cmf/schema/scm_prepare_database.sh postgresql --user=root --password=scm scm scm scm && touch /etc/cloudera-scm-server/.scm_prepare_database returned 1 instead of one of [0]
Error: /Stage[main]/Cloudera::Cm::Server/Exec[scm_prepare_database]/returns: change from notrun to 0 failed: /usr/share/cmf/schema/scm_prepare_database.sh postgresql --user=root --password=scm scm scm scm && touch /etc/cloudera-scm-server/.scm_prepare_database returned 1 instead of one of [0]
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Skipping because of failed dependencies
Debug: Servicecloudera-scm-server: Could not find cloudera-scm-server.conf in /etc/init
Debug: Servicecloudera-scm-server: Could not find cloudera-scm-server.conf in /etc/init.d
Debug: Servicecloudera-scm-server: Could not find cloudera-scm-server in /etc/init
Debug: Executing '/etc/init.d/cloudera-scm-server status'
Debug: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Skipping restart; service is not running
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Triggered 'refresh' from 1 events
Debug: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: The container Class[Cloudera::Cm::Server] will propagate my refresh event
Debug: Class[Cloudera::Cm::Server]: The container Stage[main] will propagate my refresh event
Notice: /Stage[main]/Cloudera::Cdh::Hue/Package[hue]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Cdh::Hue/Package[hue]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cdh::Mahout/Package[mahout]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Cdh::Mahout/Package[mahout]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Gplextras/Package[hadoop-lzo-cdh4]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Gplextras/Package[hadoop-lzo-cdh4]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Gplextras/Package[hadoop-lzo-cdh4-mr1]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Gplextras/Package[hadoop-lzo-cdh4-mr1]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Gplextras/Package[impala-lzo]: Dependency Exec[scm_prepare_database] has failures: true
Warning: /Stage[main]/Cloudera::Gplextras/Package[impala-lzo]: Skipping because of failed dependencies
Debug: Executing 'diff -u /etc/ganglia/gmetad.conf /tmp/puppet-file20140305-6649-ch31fj-0'
Notice: /Stage[main]/Ganglia::Gmetad::Config/File[/etc/ganglia/gmetad.conf]/content:
--- /etc/ganglia/gmetad.conf 2012-11-14 22:42:29.000000000 +0000
...
To my understanding, Cloudera manager server and Clouera manager agent are two independent roles, they should be two classes. In current code, class cloudera::cm5 is actually to install the agent ,why not rename class cloudera::cm5 to class cloudera::cm5::agent and delete the parameter install_server? I think this can enhance the readability of your code.
I get a
Jul 17 17:36:26 kmobd2-nnode2 puppet-agent[1201]: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Unknown variable: '::cloudera_cdh_reposerver'. at /etc/puppetlabs/code/environments/production/modules/cloudera/manifests/params.pp:24:21 on node ..
The reason is the strict_variable option
https://docs.puppet.com/puppet/5.0/deprecated_language.html#non-strict-variables
You should adjust your module to be able to run even with
strict_variable = true
Hi,
I can't use mysql as database, i've got the following error:
err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Mysql::Server]' for relationship on 'Exec[scm_prepare_database]' on node node-00001.mycluster.lan
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
mysql is running on node-00001.mycluster.lan
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 2476/mysqld
puppetlabs-mysql is installed on the puppetmaster:
/etc/puppet/modules
├── darin-zypprepo (v1.0.1)
├── dhcp-server (???)
├── domcleal-augeasproviders (v1.1.0)
├── mysql-server (???)
├── nanliu-staging (v0.4.0)
├── ntp (???)
├── opentsdb2 (???)
├── puppetlabs-apt (v1.4.2)
├── puppetlabs-firewall (v1.1.0)
├── puppetlabs-java_ks (v1.2.3)
├── puppetlabs-mysql (v2.2.3)
├── puppetlabs-postgresql (v2.5.0)
├── puppetlabs-stdlib (v4.2.1)
├── razorsedge-cloudera (v2.0.2)
├── ripienaar-concat (v0.2.0)
└── stahnma-epel (v0.0.6)
node define:
node 'node-00001.mycluster.lan' {
class {"cloudera":
cm_server_host => 'node-00001.mycluster.lan',
use_parcels => false,
install_cmserver => true,
install_lzo => true,
db_type => "mysql",
db_user => "root",
db_pass => $mysql_admin_password
}
}
Any idea ?
regards,
C.DUMAY
The apt sources list in /etc/apt/sources.list.d/cloudera-gplextras4.list
Will look like:
# cloudera-gplextras4
deb http://archive.cloudera.com/gplextras/debian/squeeze/amd64/gplextras/ squeeze-gplextras5 contrib
deb-src http://archive.cloudera.com/gplextras/debian/squeeze/amd64/gplextras/ squeeze-gplextras5 contrib
But it must be:
# cloudera-gplextras4
deb http://archive.cloudera.com/gplextras/debian/squeeze/amd64/gplextras/ squeeze-gplextras4 contrib
deb-src http://archive.cloudera.com/gplextras/debian/squeeze/amd64/gplextras/ squeeze-gplextras4 contrib
Hi,
Would be nice if this repo is supported by Amazon Linux too
Could be added to the list of operatingsystems ?
Thanks
It appears that many of the modules puppet-cloudera relies on are pinned to very old versions, Is there any plan to update these to more recent versions?
in particular the mysql and apt modules
I just tried to use this module on ubuntu 14.04 but it gets a 404 directly.
Error: 404 Not Found for http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key
Error: /Stage[main]/Cloudera::Cm5::Repo/Apt::Source[cloudera-manager]/Apt::Key[Add key: 327574EE02A818DD from Apt::Source cloud
era-manager]/Apt_key[Add key: 327574EE02A818DD from Apt::Source cloudera-manager]/ensure: change from absent to present failed:
404 Not Found for http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key
Notice: /Stage[main]/Cloudera::Cm5::Repo/Apt::Source[cloudera-manager]/Apt::Key[Add key: 327574EE02A818DD from Apt::Source clou
dera-manager]/Anchor[apt_key 327574EE02A818DD present]: Dependency Apt_key[Add key: 327574EE02A818DD from Apt::Source cloudera-
manager] has failures: true
Warning: /Stage[main]/Cloudera::Cm5::Repo/Apt::Source[cloudera-manager]/Apt::Key[Add key: 327574EE02A818DD from Apt::Source clo
udera-manager]/Anchor[apt_key 327574EE02A818DD present]: Skipping because of failed dependencies
I used the following in my site.pp:
node 'puppetmaster.dinglab' {
include htop
class { '::cloudera':
cm_server_host => 'puppetmaster.dinglab',
install_cmserver => true,
}
}
node 'slave1.dinglab' {
class { '::cloudera':
cm_server_host => ''puppetmaster.dinglab',
}
}
The hardcoded java version in java5.pp is jdk1.7.0.25-cloudera, while the package installed from cloudera repo is jdk1.7.0.45-cloudera
This creates incorrect symlink at /usr/java/default
Tested on centos 6.4 with cdh5 ga release.
The Cloudera Manager link in the README is down:
Hi
I am trying to deploy the cloudera using the module. I am getting the below error. I checked the same error being resolved in "#3". But still it didnt help me out.
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server' returned 1: Error Downloading Packages:
cloudera-manager-server-5.4.5-1.cm545.p0.5.el6.x86_64: failure: RPMS/x86_64/cloudera-manager-server-5.4.5-1.cm545.p0.5.el6.x86_64.rpm from cloudera-manager: [Errno 256] No more mirrors to try.
Error: /Stage[main]/Cloudera::Cm::Server/Package[cloudera-manager-server]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install cloudera-manager-server' returned 1: Error Downloading Packages:
cloudera-manager-server-5.4.5-1.cm545.p0.5.el6.x86_64: failure: RPMS/x86_64/cloudera-manager-server-5.4.5-1.cm545.p0.5.el6.x86_64.rpm from cloudera-manager: [Errno 256] No more mirrors to try.
Notice: /Stage[main]/Cloudera::Cm::Server/File[/etc/cloudera-scm-server/db.properties]: Dependency Package[cloudera-manager-server] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/File[/etc/cloudera-scm-server/db.properties]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list cloudera-manager-server-db' returned 1: Error: No matching Packages to list
Error: /Stage[main]/Cloudera::Cm::Server/Package[cloudera-manager-server-db]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y list cloudera-manager-server-db' returned 1: Error: No matching Packages to list
Notice: /Stage[main]/Cloudera::Cm::Server/Exec[cloudera-manager-server-db]: Dependency Package[cloudera-manager-server-db] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Exec[cloudera-manager-server-db]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Java/Exec[java-alternatives]/returns: executed successfully
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server-db]: Dependency Package[cloudera-manager-server-db] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server-db]: Skipping because of failed dependencies
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Dependency Package[cloudera-manager-server-db] has failures: true
Notice: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Dependency Package[cloudera-manager-server] has failures: true
Warning: /Stage[main]/Cloudera::Cm::Server/Service[cloudera-scm-server]: Skipping because of failed dependencies
Notice: Finished catalog run in 4.25 seconds
I am using this content in my site.pp
class { 'cloudera::cdh::repo':
version => '4.1.1',
}
class { 'cloudera::cm::repo':
version => '4.1.1',
}
class { 'cloudera::java': }
class { 'cloudera::cm':
server_host => 'n2.hadoop.com',
}
class { 'cloudera::cm::server': }
}
Is there any plan to update this repository to support cloudera 6? or is there any forks / new project that does ?
Hi, Thanks for the fix in 2.1.1. The _mklinks.sh script seems to generate the correct default link.
The exec commands of update-alternatives do not seem to work for me. Debugging by running by hand, it complains about the man page links. Each of the --slave options like:
--slave /usr/share/man/man1/java.1 java.1.gz /usr/java/default/man/man1/java.1
error message:
link /usr/share/man/man1/java.1.gz incorrect for slave java.1.gz (/usr/share/man/man1/java.1 java.1.gz)
only seem to work if I either add a .gz onto the first (link) parameter or remove it from the second (name) parameter. Since the command does not complete, it runs again and again for every puppet run.
Apparently the following step is needed to get the right Zookeeper package, according to http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_cdh5_install.html
The module does not do this yet, I needed to provision my server all over again because everything depends on zookeeper :(
Additional step for Trusty
This step ensures that you get the right ZooKeeper package for the current CDH release. You need to > prioritize the Cloudera repository you have just added, such that you install the CDH version of > ZooKeeper rather than the version that is bundled with Ubuntu Trusty.
To do this, create a file at /etc/apt/preferences.d/cloudera.pref with the following contents:
Package: *
Pin: release o=Cloudera, l=Cloudera
Pin-Priority: 501
The module dependencies are conflicting with the postgresql module.
puppetlabs-puppetdb depends on puppetlabs/postgresql (>= 3.1.0 <4.0.0)
puppet-cloudera depends on puppetlabs/postgresql (>=2.1.0 <3.0.0)
Could you please update the postgresql dependency?
Installing puppet-cloudera by ignoring the dependencies and manually add the modules is a workaround but a very ugly one and will mark puppetlabs-postgresql as invalid.
$ puppet module install puppetlabs-puppetdb
$ puppet module install --ignore-dependencies razorsedge-cloudera
$ puppet module install darin-zypprepo
$ puppet module install domcleal-augeasproviders
$ puppet module install nanliu-staging
$ puppet module install puppetlabs-java_ks
$ puppet module install stahnma-epel
$ puppet module install puppetlabs-mysql
$ puppet module list
Warning: Module 'puppetlabs-postgresql' (v3.3.3) fails to meet some dependencies:
'razorsedge-cloudera' (v2.0.2) requires 'puppetlabs-postgresql' (>=2.1.0 <3.0.0)
/etc/puppet/modules
├── darin-zypprepo (v1.0.1)
├── domcleal-augeasproviders (v1.1.0)
├── nanliu-staging (v0.4.0)
├── puppetlabs-apt (v1.4.2)
├── puppetlabs-concat (v1.1.0)
├── puppetlabs-firewall (v1.1.1)
├── puppetlabs-inifile (v1.0.3)
├── puppetlabs-java_ks (v1.2.3)
├── puppetlabs-mysql (v2.2.3)
├── puppetlabs-postgresql (v3.3.3) invalid
├── puppetlabs-puppetdb (v3.0.1)
├── puppetlabs-stdlib (v4.2.1)
├── razorsedge-cloudera (v2.0.2)
└── stahnma-epel (v0.0.6)
Note: I am not using postgresql as database.
Hey,
I'm running mysql locally instead of postgres and it seems that the razorsedge module executes the scm_prepare_database.sh script, so you don't actually need to invoke the script yourself.
In fact, I ran into some weird issues when calling the script myself before running the razors edge module - it seemed to cause errors within the module which caused it to return several error messages (maybe my user permissions got changed or a database got dropped, not sure).
Can you confirm that the razorsedge module actually runs the scm script itself now? Maybe it would be worth rewriting this?
If you decide not to use the embedded database, the Cloudera Manager server database configuration can be completed by configuring this module to call the scm_prepare_database.sh script. The external database must be configured and ready for connection with the supplied credentials via some method outside of this module.
Cheers,
i get the following errror
Invalid parameter architecture at /tmp/vagrant-puppet-3/modules-0/cloudera/manifests/cm5/repo.pp:142
between, is there any tutorial or an easy way to install puppet-cloudera on VM?
The following link exists in spec/fixtures/modules:
cloudera -> /home/mike/git/razorsedge/cloudera
Probably isn't supposed to be there...
Hi,
If I use this configuration:
class { 'cloudera':
install_java => true,
ensure => 'present',
cdh_version => '4',
cm_version => '4',
ci_version => '1',
cs_version => '1',
cg_version => '4',
cm_server_host => 'mainhead.es.fr.alten.com',
use_parcels => false,
}
All fails cos need install java first and this not happend.
If I use this:
class { 'cloudera::java':
ensure => 'present',
}
And after that comment and use the first configuration with install_java => false then all is ok.
I really don't understand why puppet "jump" the java installation with the main call.
Ubuntu 12, from scratch....
Hi, I'm trying to create multinode configuration. But I don't understand how to use it.
Here is my node declaration:
node 'vm-cluster-node2.localdomain' inherits default {
class { 'cloudera':
cm_version => '4.8.0',
cm_server_host => 'vm-cluster-node1.localdomain',
use_parcels => true,
}->
class { 'cloudera::cm':
server_host => 'vm-cluster-node1.localdomain',
}
}Node default doesn't have anything related to 'cloudera'. Here is an error:
Error: Duplicate declaration: Class[Cloudera::Cm] is already declared in file /tmp/vagrant-puppet/modules-0/cloudera/manifests/init.pp at line 181; cannot redeclare on node vm-cluster-node2.localdomain
What do I do wrong? I want your module to install cloudera angent on vm-cluster-node2.localdomain
Where does puppet-cloudera install the hadoop script? I can't find it on my PATH.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.