Skip to main content

Posts

Showing posts from September, 2015

Create `ext4` Partition, Format and Mount using parted (`fdisk` unable to create partition more than 2TB).

Create Partition, Format and Mounting using parted . Below is the image of how partition is divided. (courtesy from wikipedia ) Logging into the server First lets check the fdisk partition to see how much space we have on the server. Using username "root". root@192.168.100.44's password: Last login: Wed Sep 30 13:22:13 2015 from 192.168.100.2 [root@my-server ~]# fdisk -l /dev/sdb WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sdb: 13196.0 GB, 13196018581504 bytes 255 heads, 63 sectors/track, 1604324 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 1 267350 2147483647+ ee GPT Checking disk. [root@my-server ~]# df -h Filesystem

Creating a RHEL cluster with Virtual IP using CMAN and Pacemaker.

Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker. Important Links : http://blog.mattbrock.co.uk/creating-a-two-node-centos-6-cluster-with-floating-ip-using-cman-and-pacemaker/ http://clusterlabs.org/quickstart-redhat-6.html Configuring Repo on RHEL 6.6 [root@waepprrkhe001 ~]# cat /etc/yum.repos.d/centos.repo [centos-6-base] name=CentOS-$releasever - Base mirrorlist=http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os baseurl=http://mirror.centos.org/centos/6/os/x86_64/ enabled=1 gpgkey=http://mirror.centos.org/centos/6/os/x86_64/RPM-GPG-KEY-CentOS-6 [root@waepprrkhe001 ~]# Installation and initial configuration Install the required packages on both machines: yum install pacemaker cman pcs ccs resource-agents Set up and configure the cluster on the primary machine, changing vipcluster, primary.server.com and secondary.server.com as needed: ccs -f /etc/cluster/cluster.conf --createcluster vipcluster ccs -f /etc/cluster/cluster.conf

Setting up Pentaho Data Integration 5.4.1 with Hadoop

Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager) Setting up pentaho server has the below steps. Will be referring to Pentaho Data Integration as PDI from now on. Install PDI. Make ETL (PDI) as the Gateway for HDFS, HIVE, YARN, HBASE from Cloudera Manager. Test Access to Hadoop from PDI Server. Updating configuration in PDI for Hadoop Setup. Test PDI with Hadoop. Installing Pentaho Server 5.4.1 Community Edition. Installation is very straight forward, we get a pdi.zip which needs to be extracted in the location required. Here we will be extracting the file into /opt directory on our RHEL ETL PDI Server. Extract and move the directory to /opt [root@pentaho-server ~]# unzip pdi.zip [root@pentaho-server ~]# mv data-integration /opt Create a soft Link /opt directory, we have used kettle / pentaho as the softlink. [root@pentaho-server ~]# cd opt [root@pentaho-server opt]# ln -s data-integration pentaho [root@pentaho-server opt