What I See

We had recently built a cluster using cloudera API’s and had all the services running on it with Kerberos enabled. Next we had a requirement to add another kafka cluster to our already exsisting cluster in cloudera manager. Since it is a quick task to get the zookeeper and kafka up and running. We decided to get this done using the cloudera manager instead of the API’s. But we faced the Duplicate entry 'zookeeper' for key 'NAME' issue as described in the bug below. https://issues.cloudera.org/browse/DISTRO-790 I have set up two clusters that share a Cloudera Manger. The first I set up with the API and created the services with capital letter names, e.g., ZOOKEEPER, HDFS, HIVE. Now, I add the second cluster using the Wizard. Add Cluster->Select Hosts->Distribute Parcels->Select base HDFS Cluster install On the next page i get SQL errros telling that the services i want to add already exist. I suspect that the check for existing service names does n...

We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it. During installation we notice that the distribution is not working. There could be couple of reasons. This was the second time we are having this issue. Check /etc/hosts file if we have all the server names added correctly Second reason would be due to the fact that, one of the installation was terminated midway, leaving a stale config which set the status to ACTIVATING for CDH parcel. So when we try to install, parcel was not distributing. Again there could be similar issue if we do not have enough space on the node for /opt/cloudera . Solution: Deactivating parcel and retry. curl -u username:password -X POST http://adminnode: 7180 /api/v14/clusters/ /parcels/products/CDH/versions/ 5.10 .0 - 1. cdh5 .10 .0 .p0 .41 /commands/deactivate Check for space and increase space for /opt/cloudera/ Most of the time should see th...

Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using Chef/Ansible, but cloudera API give more control over the cluster. One of the awesome features of cloudera API is to setup kerberos for the cluster, which otherwise done manually (command line) is very tricky task (loads of this to go wrong). For our kerberos setup we will need to complete below steps. Cloudera Manager Configuration. Stop Cluster. Stop Cloudera Manager service. Service configuration for all the services which needs updates like HDFS , HBASE , KAFKA , ZOOKEEPER , HUE . Creating KT_RENEWER for HUE service. Start Cloudera Manager Services. Start Cluster. Steps To kerberize Cloudera Cluster Step 1: Configure cloudera with LDAP information. # # Deploy LDAP configuration for CM # self.deploy_cm_ldap_configuration() Step 2: Import Admin credentials, using the username / password, which h...