Now that all the prerequisites have been performed in Part 1, we can jump in the Exadata installation procedure itself.
All the installation steps are performed by a script named install.sh (provided by OEDA) with the help of the well known <client>-<cluster>.xml XML file. It is quite straightforward. All steps can be launched in one command but I recommend launching them step by step, it is easier to troubleshoot if needed. One important thing is to not jump over a failed step; resolve the failed step first and then continue.
The install.sh script we will be using is located in the /u01/Exadata_software/linux-x64/ directory as well as the <client>-<cluster>.xml configuration file (see in Part 1 for more detail on this).
4.1/ Step 1
As the install.sh script is meant for many different configurations, the step 1 will perform some checks and adapt the list of steps that are needed for your particular configuration.
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 1 Initializing Executing Validate Configuration File Validating cluster: mycluster Locating machines... Verifying operating systems... Validating cluster networks... Validating network connectivity... Validating NTP setup... Validating physical disks on storage cells... Validating users... Validating platinum... Validating switches... Checking disk reclaim status... Checking Disk Tests Status.... Completed validation... …Check the output of the script to see if anything is wrong and we can then continue.
4.1.2/ List of Steps
The -l option can be used to list all those steps now that the step 1 has adapted it to your configuration.
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -l Initializing 1. Validate Configuration File 2. Setup Required Files 3. Create Users 4. Setup Cell Connectivity 5. Verify Infiniband 6. Calibrate Cells 7. Create Cell Disks 8. Create Grid Disks 9. Install Cluster Software 10. Initialize Cluster Software 11. Install Database Software 12. Relink Database with RDS 13. Create ASM Diskgroups 14. Create Databases 15. Apply Security Fixes 16. Install Exachk 17. Create Installation Summary 18. Resecure Machine [root@myclusterdb01 linux-x64]#
4.2/ Step 2
This step will check for all the ISO files needed for all the installation steps, you will be prompted for missing files; you would then have to download and copy the missing files in the WorkDir directory and relaunch the step.
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 2 Initializing Executing Setup Required Files Copying and extracting required files... Required files are: /u01/Exadata_software/linux-x64/WorkDir/linuxamd64_12102_grid_1of2.zip /u01/Exadata_software/linux-x64/WorkDir/linuxamd64_12102_grid_2of2.zip … Copying file: p6880880_121010_Linux-x86-64.zip to node myclusterdb04 Completed copying files... Extracting required files... Copying resourcecontrol and other required files Creating databasemachine.xml for EM discovery Done Creating databasemachine.xml for EM discovery Successfully completed execution of step Setup Required Files [elapsed Time [Elapsed = 345038 mS [5.0 minutes] Tue Apr 05 21:30:32 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.3/ Step 3
This step will create the required users
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 3 Initializing Executing Create Users Creating users... Creating users in cluster mycluster Validating existing users and groups... Creating required directories on nodes in cluster mycluster Updating /etc/hosts on nodes in cluster mycluster Setting up ssh for users in cluster mycluster Completed creating all users... Successfully completed execution of step Create Users [elapsed Time [Elapsed = 113443 mS [1.0 minutes] Tue Apr 05 19:55:36 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.4/ Step 4
This step will set up the cell connectivity creating the /etc/oracle/cell/network-config/cellip.ora and /etc/oracle/cell/network-config/cellinit.ora files
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 4 Initializing Executing Setup Cell Connectivity Creating cellip.ora and cellinit.ora ... Creating cellip.ora for cluster mycluster Creating cellinit.ora for cluster mycluster Creating cellAffinity for cluster mycluster Creating cellaffinity if required Creating cellaffinity is NOT required Creating cellaffinity is NOT required Creating cellaffinity is NOT required Creating cellaffinity is NOT required Done with cellaffinity.ora... Done creating cellip.ora and cellinit.ora... Successfully completed execution of step Setup Cell Connectivity [elapsed Time [Elapsed = 26897 mS [0.0 minutes] Tue Apr 05 19:57:27 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.5/ Step 5
This step will check the Infiniband configuration and test the performances of it using the infinicheck script
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 5 Initializing Executing Verify Infiniband Validating infiniband network with rds-ping... Check Admin networks connection... Running infinicheck to verify infiniband fabric for cluster mycluster... Checking status of infiniband links in cluster mycluster... Running verify topology to verify infiniband network... No Infiniband link errors found... SUCCESS: Verify topology does not report any errors on node myclusterdb01 ... … Successfully completed execution of step Verify Infiniband [elapsed Time [Elapsed = 823374 mS [13.0 minutes] Mon Apr 18 18:14:06 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.6/ Step 6
This step will check the cells using the cellcli -e calibrate command. It also tests the performance of the cell disks and report any issue
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 6 Initializing Executing Calibrate Cells Calibrating cells... Successfully completed execution of step Calibrate Cells [elapsed Time [Elapsed = 471323 mS [7.0 minutes] Tue Apr 05 21:06:39 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.7/ Step 7
This step will create the cell disks, flashlog and flashcache
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 7 Initializing Executing Create Cell Disks Checking physical disks for errors before creating celldisks Restarting cell services... Initializing cells... Cleaning cells... Restarting cell services... Creating cell disks... Creating flashlog on cells... Creating flashcache on cells... Successfully completed execution of step Create Cell Disks [elapsed Time [Elapsed = 125431 mS [2.0 minutes] Tue Apr 05 21:11:25 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.8/ Step 8
This step will create the grid disks
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 8 Initializing Executing Create Grid Disks Creating grid disks for cluster mycluster Successfully completed execution of step Create Grid Disks [elapsed Time [Elapsed = 32878 mS [0.0 minutes] Tue Apr 05 21:15:07 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.9/ Step 9
This step will install the Grid Infrastructure 12.1.0.2
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 9 Initializing Executing Install Cluster Software Installing cluster mycluster Getting grid disks using utility in /u01/app/oracle/Oeda/Software/12.1.0.2/grid Writing grid response file for cluster mycluster Running clusterware installer... Setting up Opatch for cluster mycluster Patching cluster mycluster... Successfully completed execution of step Install Cluster Software [elapsed Time [Elapsed = 945400 mS [15.0 minutes] Wed Apr 13 22:37:02 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.10/ Step 10
This step will run the root.sh script on all the nodes.
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 10 Initializing Executing Initialize Cluster Software Initializing cluster mycluster Relinking with RDS protocol... Running root.sh on node myclusterdb01.mydomain.com Checking file root_myclusterdb01.mydomain.com_2016-04-13_22-38-09.log on node myclusterdb01.mydomain.com Running root.sh on node myclusterdb02.mydomain.com Checking file root_myclusterdb02.mydomain.com_2016-04-13_22-46-51.log on node myclusterdb02.mydomain.com Running root.sh on node myclusterdb03.mydomain.com Checking file root_myclusterdb03.mydomain.com_2016-04-13_22-54-03.log on node myclusterdb03.mydomain.com Running root.sh on node myclusterdb04.mydomain.com Checking file root_myclusterdb04.mydomain.com_2016-04-13_23-01-13.log on node myclusterdb04.mydomain.com Running configuration assistants... Running configuration tools on machine myclusterdb01.mydomain.com Checking status of cluster... Completed cluster initialization... Successfully completed execution of step Initialize Cluster Software [elapsed Time [Elapsed = 2572955 mS [42.0 minutes] Wed Apr 13 23:20:44 CDT 2016]] [root@myclusterdb01 linux-x64]#Note : if any issue occurs during this step, you will have to deinstall the GI:
$ /u01/app/oracle/product/12.1.0.2/dbhome_1/deinstall/deinstall # dcli -g /root/dbs_group -l root "rm -fr /u01/app/12.1.0.2/grid" # dcli -g /root/dbs_group -l root "mv /u01/app/oraInventory /u01/app/oraInventory.old"Then reinstall the GI (step 9) and relaunch root.sh (step 10).
4.11/ Step 11
This step will install Oracle database -- this was 11.2.0.4 in this case (not 12c, I installed it manually later on)
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 11 Initializing Executing Install Database Software Installing database software ... Installing database home software, database home c0_DbHome_0 Installing database software ... Running database installer on node myclusterdb01.mydomain.com ... Please wait... After running database installer... Successfully completed execution of step Install Database Software [elapsed Time [Elapsed = 849646 mS [14.0 minutes] Wed Apr 13 23:45:05 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.12/ Step 12
This step will relink the installed Oracle databases homes with RDS (protocol to use Infiniband)
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 12 Initializing Executing Relink Database with RDS Successfully completed execution of step Relink Database with RDS [elapsed Time [Elapsed = 7050 mS [0.0 minutes] Wed Apr 13 23:47:03 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.13/ Step 13
This step will create the ASM diskgroups according to what has been defined by the client in the XML configuration file (this is the perfect timing to double check, triple check, etc... the name of the diskgroups before creating it to avoid to have to rename the diskgroups later)
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 13 Initializing Executing Create ASM Diskgroups Getting grid disks using utility in /u01/app/oracle/Oeda/Software/12.1.0.2/grid Getting grid disks using utility in /u01/app/oracle/Oeda/Software/12.1.0.2/grid Successfully completed execution of step Create ASM Diskgroups [elapsed Time [Elapsed = 198770 mS [3.0 minutes] Thu Apr 14 20:52:39 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.14/ Step 14
This step will create the databases as defined in the <client>-<cluster>.xml file. In my case, no database was defined there as the client wanted to create them manually.
But you can easily do it like that :
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 14 .... [root@myclusterdb01 linux-x64]#
4.15/ Step 15
This step applies some security fixes; the error below is due to the fact that the Step 14 has not been executed; it can then be ignored
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 15 Initializing Executing Apply Security Fixes Checking database dbm01 status on cluster mycluster Database instance dbm01 in cluster mycluster is not Running. Restart databases and re-run step...! ERROR: Database instance dbm01 in cluster mycluster is not Running. Restart databases and re-run step...! Collecting diagnostics... Errors occurred. Send /u01/Exadata_software/linux-x64/WorkDir/Diag-160414_215154.zip to Oracle to receive assistance. ERROR: Errors occured... Errors occured, exiting... [root@myclusterdb01 linux-x64]#
4.16/ Step 16
This step installs the Exachk tool
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 16 Initializing Executing Install Exachk Installing Exachk Exachk has been installed on compute node: mycluster2db01.mydomain.com at: /opt/oracle.SupportTools/exachk and can be run by invoking ./exachk Successfully completed execution of step Install Exachk [elapsed Time [Elapsed = 5132 mS [0.0 minutes] Thu Apr 14 21:55:35 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.17/ Step 17
This step creates an installation summary
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 17 Initializing Executing Create Installation Summary Getting system details... Running verify topology to verify infiniband network... Validating infiniband network with rds-ping... Check Admin networks connection... Running infinicheck to verify infiniband fabric for cluster mycluster... Checking status of infiniband links in cluster mycluster... Running verify topology to verify infiniband network... No Infiniband link errors found... … Installation summary report: /u01/Exadata_software/linux-x64/ExadataConfigurations/Client-AMI_Dev_Exadata-InstallationReport.xml Successfully completed execution of step Create Installation Summary [elapsed Time [Elapsed = 877799 mS [14.0 minutes] Thu Apr 14 22:12:13 CDT 2016]] [root@myclusterdb01 linux-x64]#
4.18/ Step 18
This step is to “Resecure the machine”; this step is meant to :
- expire all passwords (need to be changed at next login)
- implement password aging and complexity
- tightened some permissions
- drop the root ssh keys
[root@myclusterdb01 linux-x64]# ./install.sh -cf client-mycluster.xml -s 18 ... [root@myclusterdb01 linux-x64]#
You are now done, and everything is now installed properly. You can now create many databases and start to enjoy Exadata !
Excellent material. Congrats and thanks for sharing.
ReplyDelete