I have a customer that needs to migrate its Oracle RAC cluster from SuSE to OEL.<\/p>\n
I know, I know, there is a paper from Dell and Oracle named:<\/p>\n
That explains how Dell migrated its many RAC clusters from SuSE to OEL. The problem is that they used a different strategy:<\/p>\n
– backup the configuration of the nodes
\n– then for each node, one at time
\n– stop the node
\n– reinstall the OS
\n– restore the configuration and the Oracle binaries
\n– relink
\n– restart<\/p>\n
What I want to achieve instead is: because the customer will also migrate to new hardware.<\/p>\n In order to test this migration path, I’ve set up a SINGLE NODE cluster (if it works for one node, it will for two or more).<\/p>\n I\u00a0have to setup the new node addition carefully, mainly as I\u00a0would do with a traditional node addition:<\/p>\n Once the new host ready, the cluvfy stage -pre nodeadd will likely fail due to<\/p>\n Here’s an example of output:<\/p>\n So the problem is not if<\/strong> the check succeed or not (it will not), but what<\/strong> fails.<\/p>\n Solving all the problems not related to the difference SuSE-OEL is crucial, because the addNode.sh will fail<\/strong> with the same errors.\u00a0\u00a0I\u00a0need to run it using -ignorePrereqs and -ignoreSysPrereqs switches. Let’s see how it\u00a0works:<\/p>\n Then, as stated by the addNode.sh, I run the root.sh and I expect it to work:<\/p>\n Bingo! Let’s check if everything is up and running:<\/p>\n So yes, it works, but remember that it’s not a supported long-term configuration.<\/p>\n In my case I expect to migrate the whole cluster from SLES to OEL in one day.<\/p>\n NOTE: using OEL6 as new target is easy because the interface names do not change. The new OEL7 interface naming changes, if you need to migrate without cluster downtime you need to setup the new OEL7 nodes following this post:\u00a0http:\/\/ask.xmodulo.com\/change-network-interface-name-centos7.html<\/a><\/p>\n Otherwise, you need to configure a new interface name for the cluster with oifcfg.<\/p>\n HTH<\/p>\n —<\/p>\n Ludovico<\/p>\n","protected":false},"excerpt":{"rendered":" I have a customer that needs to migrate its Oracle RAC cluster from SuSE to OEL. I know, I know, there is a paper from Dell and Oracle named: How Dell Migrated from SUSE Linux to Oracle Linux That explains … Continue reading
\n– add one OEL node to the SuSE cluster as new node<\/strong>
\n– remove one SuSE node from the now-mixed cluster
\n– install\/restore\/relink the RDBMS software (RAC) on the new node
\n–\u00a0move the RAC instances to the new node (taking care to NOT run more than the number of licensed nodes\/CPUs at any time)
\n– repeat (for the remaining\u00a0nodes)<\/p>\noracle@sles01:~> crsctl stat res -t\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nLocal Resources\r\n--------------------------------------------------------------------------------\r\nora.DATA.dg\r\n ONLINE ONLINE sles01 STABLE\r\nora.LISTENER.lsnr\r\n ONLINE ONLINE sles01 STABLE\r\nora.asm\r\n ONLINE ONLINE sles01 Started,STABLE\r\nora.net1.network\r\n ONLINE ONLINE sles01 STABLE\r\nora.ons\r\n ONLINE ONLINE sles01 STABLE\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.LISTENER_SCAN1.lsnr\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.cvu\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.oc4j\r\n 1 OFFLINE OFFLINE STABLE\r\nora.scan1.vip\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.sles01.vip\r\n 1 ONLINE ONLINE sles01 STABLE\r\n--------------------------------------------------------------------------------\r\noracle@sles01:~> cat \/etc\/issue\r\n\r\nWelcome to SUSE Linux Enterprise Server 11 SP4 (x86_64) - Kernel \\r (\\l).\r\n<\/pre>\n
\n
\n
oracle@sles01:~> cluvfy stage -pre nodeadd -n rhel01\r\n\r\nPerforming pre-checks for node addition\r\n\r\nChecking node reachability...\r\nNode reachability check passed from node \"sles01\"\r\n\r\n\r\nChecking user equivalence...\r\nUser equivalence check passed for user \"oracle\"\r\nPackage existence check passed for \"cvuqdisk\"\r\n\r\nChecking CRS integrity...\r\n\r\nCRS integrity check passed\r\n\r\nClusterware version consistency passed.\r\n\r\nChecking shared resources...\r\n\r\nChecking CRS home location...\r\nLocation check passed for: \"\/u01\/app\/12.1.0\/grid\"\r\nShared resources check for node addition passed\r\n\r\n\r\nChecking node connectivity...\r\n\r\nChecking hosts config file...\r\n\r\nVerification of the hosts config file successful\r\n\r\nCheck: Node connectivity using interfaces on subnet \"192.168.56.0\"\r\nNode connectivity passed for subnet \"192.168.56.0\" with node(s) sles01,rhel01\r\nTCP connectivity check passed for subnet \"192.168.56.0\"\r\n\r\n\r\nCheck: Node connectivity using interfaces on subnet \"172.16.100.0\"\r\nNode connectivity passed for subnet \"172.16.100.0\" with node(s) rhel01,sles01\r\nTCP connectivity check passed for subnet \"172.16.100.0\"\r\n\r\nChecking subnet mask consistency...\r\nSubnet mask consistency check passed for subnet \"192.168.56.0\".\r\nSubnet mask consistency check passed for subnet \"172.16.100.0\".\r\nSubnet mask consistency check passed.\r\n\r\nNode connectivity check passed\r\n\r\nChecking multicast communication...\r\n\r\nChecking subnet \"172.16.100.0\" for multicast communication with multicast group \"224.0.0.251\"...\r\nCheck of subnet \"172.16.100.0\" for multicast communication with multicast group \"224.0.0.251\" passed.\r\n\r\nCheck of multicast communication passed.\r\nTotal memory check passed\r\nAvailable memory check passed\r\nSwap space check passed\r\nFree disk space check passed for \"sles01:\/usr,sles01:\/var,sles01:\/etc,sles01:\/u01\/app\/12.1.0\/grid,sles01:\/sbin,sles01:\/tmp\"\r\nFree disk space check passed for \"rhel01:\/usr,rhel01:\/var,rhel01:\/etc,rhel01:\/u01\/app\/12.1.0\/grid,rhel01:\/sbin,rhel01:\/tmp\"\r\nCheck for multiple users with UID value 1101 passed\r\nUser existence check passed for \"oracle\"\r\nRun level check passed\r\nHard limits check passed for \"maximum open file descriptors\"\r\nSoft limits check passed for \"maximum open file descriptors\"\r\nHard limits check passed for \"maximum user processes\"\r\nSoft limits check passed for \"maximum user processes\"\r\nSystem architecture check passed\r\n\r\nWARNING:\r\nPRVF-7524 : Kernel version is not consistent across all the nodes.\r\nKernel version = \"3.0.101-63-default\" found on nodes: sles01.\r\nKernel version = \"3.8.13-16.2.1.el6uek.x86_64\" found on nodes: rhel01.\r\nKernel version check passed\r\nKernel parameter check passed for \"semmsl\"\r\nKernel parameter check passed for \"semmns\"\r\nKernel parameter check passed for \"semopm\"\r\nKernel parameter check passed for \"semmni\"\r\nKernel parameter check passed for \"shmmax\"\r\nKernel parameter check passed for \"shmmni\"\r\nKernel parameter check passed for \"shmall\"\r\nKernel parameter check passed for \"file-max\"\r\nKernel parameter check passed for \"ip_local_port_range\"\r\nKernel parameter check passed for \"rmem_default\"\r\nKernel parameter check passed for \"rmem_max\"\r\nKernel parameter check passed for \"wmem_default\"\r\nKernel parameter check passed for \"wmem_max\"\r\nKernel parameter check passed for \"aio-max-nr\"\r\nPackage existence check passed for \"make\"\r\nPackage existence check passed for \"libaio\"\r\nPackage existence check passed for \"binutils\"\r\nPackage existence check passed for \"gcc(x86_64)\"\r\nPackage existence check passed for \"gcc-c++(x86_64)\"\r\nPackage existence check passed for \"glibc\"\r\nPackage existence check passed for \"glibc-devel\"\r\nPackage existence check passed for \"ksh\"\r\nPackage existence check passed for \"libaio-devel\"\r\nPackage existence check failed for \"libstdc++33\"\r\nCheck failed on nodes:\r\n rhel01\r\nPackage existence check failed for \"libstdc++43-devel\"\r\nCheck failed on nodes:\r\n rhel01\r\nPackage existence check passed for \"libstdc++-devel(x86_64)\"\r\nPackage existence check failed for \"libstdc++46\"\r\nCheck failed on nodes:\r\n rhel01\r\nPackage existence check failed for \"libgcc46\"\r\nCheck failed on nodes:\r\n rhel01\r\nPackage existence check passed for \"sysstat\"\r\nPackage existence check failed for \"libcap1\"\r\nCheck failed on nodes:\r\n rhel01\r\nPackage existence check failed for \"nfs-kernel-server\"\r\nCheck failed on nodes:\r\n rhel01\r\nCheck for multiple users with UID value 0 passed\r\nCurrent group ID check passed\r\n\r\nStarting check for consistency of primary group of root user\r\n\r\nCheck for consistency of root user's primary group passed\r\nGroup existence check passed for \"asmadmin\"\r\nGroup existence check passed for \"asmoper\"\r\nGroup existence check passed for \"asmdba\"\r\n\r\nChecking ASMLib configuration.\r\nCheck for ASMLib configuration passed.\r\n\r\nChecking OCR integrity...\r\n\r\nOCR integrity check passed\r\n\r\nChecking Oracle Cluster Voting Disk configuration...\r\n\r\nOracle Cluster Voting Disk configuration check passed\r\nTime zone consistency check passed\r\n\r\nStarting Clock synchronization checks using Network Time Protocol(NTP)...\r\n\r\nNTP Configuration file check started...\r\nNo NTP Daemons or Services were found to be running\r\n\r\nClock synchronization check using Network Time Protocol(NTP) passed\r\n\r\n\r\nUser \"oracle\" is not part of \"root\" group. Check passed\r\nChecking integrity of file \"\/etc\/resolv.conf\" across nodes\r\n\r\n\"domain\" and \"search\" entries do not coexist in any \"\/etc\/resolv.conf\" file\r\nAll nodes have same \"search\" order defined in file \"\/etc\/resolv.conf\"\r\nPRVF-5636 : The DNS response time for an unreachable node exceeded \"15000\" ms on following nodes: sles01,rhel01\r\n\r\nCheck for integrity of file \"\/etc\/resolv.conf\" failed\r\n\r\n\r\nChecking integrity of name service switch configuration file \"\/etc\/nsswitch.conf\" ...\r\nCheck for integrity of name service switch configuration file \"\/etc\/nsswitch.conf\" passed\r\n\r\n\r\nPre-check for node addition was unsuccessful on all the nodes.\r\n<\/pre>\n
oracle@sles01:\/u01\/app\/12.1.0\/grid\/addnode> .\/addnode.sh -silent \"CLUSTER_NEW_NODES={rhel01}\" \"CLUSTER_NEW_VIRTUAL_HOSTNAMES={rhel01-vip}\" -ignorePrereq -ignoreSysPrereqs\r\nStarting Oracle Universal Installer...\r\n\r\nChecking Temp space: must be greater than 120 MB. Actual 27479 MB Passed\r\nChecking swap space: must be greater than 150 MB. Actual 2032 MB Passed\r\n\r\nPrepare Configuration in progress.\r\n\r\nPrepare Configuration successful.\r\n.................................................. 9% Done.\r\nYou can find the log of this install session at:\r\n \/u01\/app\/oraInventory\/logs\/addNodeActions2015-11-09_09-57-16PM.log\r\n\r\nInstantiate files in progress.\r\n\r\nInstantiate files successful.\r\n.................................................. 15% Done.\r\n\r\nCopying files to node in progress.\r\n\r\nCopying files to node successful.\r\n.................................................. 79% Done.\r\n\r\nSaving cluster inventory in progress.\r\n.................................................. 87% Done.\r\n\r\nSaving cluster inventory successful.\r\nThe Cluster Node Addition of \/u01\/app\/12.1.0\/grid was successful.\r\nPlease check '\/tmp\/silentInstall.log' for more details.\r\n\r\nAs a root user, execute the following script(s):\r\n 1. \/u01\/app\/oraInventory\/orainstRoot.sh\r\n 2. \/u01\/app\/12.1.0\/grid\/root.sh\r\n\r\nExecute \/u01\/app\/oraInventory\/orainstRoot.sh on the following nodes:\r\n[rhel01]\r\nExecute \/u01\/app\/12.1.0\/grid\/root.sh on the following nodes:\r\n[rhel01]\r\n\r\nThe scripts can be executed in parallel on all the nodes. If there are any policy managed databases managed by cluster, proceed with the addnode procedure without executing the root.sh script. Ensure that root.sh script is executed after all the policy managed databases managed by clusterware are extended to the new nodes.\r\n..........\r\nUpdate Inventory in progress.\r\n.................................................. 100% Done.\r\n\r\nUpdate Inventory successful.\r\nSuccessfully Setup Software.\r\n<\/pre>\n[oracle@rhel01 install]$ sudo \/u01\/app\/12.1.0\/grid\/root.sh\r\nPerforming root user operation for Oracle 12c\r\n\r\nThe following environment variables are set as:\r\n ORACLE_OWNER= oracle\r\n ORACLE_HOME= \/u01\/app\/12.1.0\/grid\r\n Copying dbhome to \/usr\/local\/bin ...\r\n Copying oraenv to \/usr\/local\/bin ...\r\n Copying coraenv to \/usr\/local\/bin ...\r\n\r\nEntries will be added to the \/etc\/oratab file as needed by\r\nDatabase Configuration Assistant when a database is created\r\nFinished running generic part of root script.\r\nNow product-specific root actions will be performed.\r\nRelinking oracle with rac_on option\r\nUsing configuration parameter file: \/u01\/app\/12.1.0\/grid\/crs\/install\/crsconfig_params\r\n2015\/11\/09 23:18:42 CLSRSC-363: User ignored prerequisites during installation\r\n\r\nOLR initialization - successful\r\n2015\/11\/09 23:19:08 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.conf'\r\n\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\nCRS-4123: Oracle High Availability Services has been started.\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\nCRS-4123: Oracle High Availability Services has been started.\r\nCRS-4133: Oracle High Availability Services has been stopped.\r\nCRS-4123: Starting Oracle High Availability Services-managed resources\r\nCRS-2672: Attempting to start 'ora.mdnsd' on 'rhel01'\r\nCRS-2672: Attempting to start 'ora.evmd' on 'rhel01'\r\nCRS-2676: Start of 'ora.mdnsd' on 'rhel01' succeeded\r\nCRS-2676: Start of 'ora.evmd' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.gpnpd' on 'rhel01'\r\nCRS-2676: Start of 'ora.gpnpd' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.gipcd' on 'rhel01'\r\nCRS-2676: Start of 'ora.gipcd' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.cssdmonitor' on 'rhel01'\r\nCRS-2676: Start of 'ora.cssdmonitor' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.cssd' on 'rhel01'\r\nCRS-2672: Attempting to start 'ora.diskmon' on 'rhel01'\r\nCRS-2676: Start of 'ora.diskmon' on 'rhel01' succeeded\r\nCRS-2789: Cannot stop resource 'ora.diskmon' as it is not running on server 'rhel01'\r\nCRS-2676: Start of 'ora.cssd' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rhel01'\r\nCRS-2672: Attempting to start 'ora.ctssd' on 'rhel01'\r\nCRS-2676: Start of 'ora.ctssd' on 'rhel01' succeeded\r\nCRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.asm' on 'rhel01'\r\nCRS-2676: Start of 'ora.asm' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.storage' on 'rhel01'\r\nCRS-2676: Start of 'ora.storage' on 'rhel01' succeeded\r\nCRS-2672: Attempting to start 'ora.crsd' on 'rhel01'\r\nCRS-2676: Start of 'ora.crsd' on 'rhel01' succeeded\r\nCRS-6017: Processing resource auto-start for servers: rhel01\r\nCRS-2672: Attempting to start 'ora.ons' on 'rhel01'\r\nCRS-2676: Start of 'ora.ons' on 'rhel01' succeeded\r\nCRS-6016: Resource auto-start has completed for server rhel01\r\nCRS-6024: Completed start of Oracle Cluster Ready Services-managed resources\r\nCRS-4123: Oracle High Availability Services has been started.\r\n2015\/11\/09 23:22:06 CLSRSC-343: Successfully started Oracle clusterware stack\r\n\r\nclscfg: EXISTING configuration version 5 detected.\r\nclscfg: version 5 is 12c Release 1.\r\nSuccessfully accumulated necessary OCR keys.\r\nCreating OCR keys for user 'root', privgrp 'root'..\r\nOperation successful.\r\nPreparing packages for installation...\r\ncvuqdisk-1.0.9-1\r\n2015\/11\/09 23:22:23 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded\r\n<\/pre>\n
[oracle@rhel01 ~]$ \/u01\/app\/12.1.0\/grid\/bin\/crsctl stat res -t\r\n--------------------------------------------------------------------------------\r\nName Target State Server State details\r\n--------------------------------------------------------------------------------\r\nLocal Resources\r\n--------------------------------------------------------------------------------\r\nora.DATA.dg\r\n ONLINE ONLINE rhel01 STABLE\r\n ONLINE ONLINE sles01 STABLE\r\nora.LISTENER.lsnr\r\n ONLINE ONLINE rhel01 STABLE\r\n ONLINE ONLINE sles01 STABLE\r\nora.asm\r\n ONLINE ONLINE rhel01 Started,STABLE\r\n ONLINE ONLINE sles01 Started,STABLE\r\nora.net1.network\r\n ONLINE ONLINE rhel01 STABLE\r\n ONLINE ONLINE sles01 STABLE\r\nora.ons\r\n ONLINE ONLINE rhel01 STABLE\r\n ONLINE ONLINE sles01 STABLE\r\n--------------------------------------------------------------------------------\r\nCluster Resources\r\n--------------------------------------------------------------------------------\r\nora.LISTENER_SCAN1.lsnr\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.cvu\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.oc4j\r\n 1 OFFLINE OFFLINE STABLE\r\nora.rhel01.vip\r\n 1 ONLINE ONLINE rhel01 STABLE\r\nora.scan1.vip\r\n 1 ONLINE ONLINE sles01 STABLE\r\nora.sles01.vip\r\n 1 ONLINE ONLINE sles01 STABLE\r\n--------------------------------------------------------------------------------\r\n<\/pre>\n
[oracle@rhel01 ~]$ olsnodes -s\r\nsles01 Active\r\nrhel01 Active\r\n\r\n[oracle@rhel01 ~]$ ssh rhel01 uname -r\r\n3.8.13-16.2.1.el6uek.x86_64\r\n[oracle@rhel01 ~]$ ssh sles01 uname -r\r\n3.0.101-63-default\r\n\r\n[oracle@rhel01 ~]$ ssh rhel01 cat \/etc\/redhat-release\r\nRed Hat Enterprise Linux Server release 6.5 (Santiago)\r\n[oracle@rhel01 ~]$ ssh sles01 cat \/etc\/issue\r\nWelcome to SUSE Linux Enterprise Server 11 SP4 (x86_64) - Kernel \\r (\\l).<\/pre>\n