The Exadata compute nodes and storage servers are some Linux servers with a specific software which optimizes everything for the Oracle database, the Oracle clusterware, etc... it means that the software is a mix of RPMs like any other Red Hat system and when we patch these servers, Oracle provides an ISO (in note 888828.1) containing the new version of all these RPMs and patchmgr updates all these RPMs. And usually, our job stops here.
But . . . it may happen that a RPM needs to be reinstalled as it may be become corrupted or broken. Let's take as an example the Exadata Management Package which provides the ability to monitor the Exadata compute nodes and storage cells. This Exadata Management Package software is located in /etc/oracle/dbserver so if you remove this directory (don't do that :)), the dbmcli command won't work any more and you will be in trouble (for patching and more).
The situation will then be that the RPM wil still be installed at the system level but dbmcli is not here any more:
To get the ISO out of the patch, you need to unzip the patch (thing that patchmgr does for us when patching):
The important thing to understand here is that the patches provided in the note 888828.1 to patch your Exadatas servers are ISO containing RPMs which can then be used to reinstall some RPMs in case of something wrong happens on your systems -- and this can be very very useful !
But . . . it may happen that a RPM needs to be reinstalled as it may be become corrupted or broken. Let's take as an example the Exadata Management Package which provides the ability to monitor the Exadata compute nodes and storage cells. This Exadata Management Package software is located in /etc/oracle/dbserver so if you remove this directory (don't do that :)), the dbmcli command won't work any more and you will be in trouble (for patching and more).
The situation will then be that the RPM wil still be installed at the system level but dbmcli is not here any more:
[root@node01]# rpm -qa | grep exadata-dbmmgmt exadata-dbmmgmt-19.2.8.0.0.191119-1.noarch [root@node01]# dbmcli dbmcli: command not foundTo fix this situation, you need to reinstall this RPM and to start with, you need to get it for your current version (imageinfo or exa-versions.sh will give you the current version of your system). If you have patched your Exadata yourself, you will have the patch already somewhere; if not, you'll find it in note 888828.1.
To get the ISO out of the patch, you need to unzip the patch (thing that patchmgr does for us when patching):
[root@node01]# unzip -q p30398147_192000_Linux-x86-64.zip [root@node01]# ls -ltr exa* -rw-rw-r-- 1 root root 1421932544 Nov 20 2019 exadata_ol7_base_repo_19.2.8.0.0.191119.isoNow, we will mount the ISO:
[root@node01]# mkdir /mnt [root@node01]# mount -o loop exadata_ol7_base_repo_19.2.8.0.0.191119.iso /mnt mount: /dev/loop0 is write-protected, mounting read-onlyWe can now copy the RPM somewhere for future use and umount the ISO (we are done with it, you can delete it):
[root@node01]# cp /mnt/x86_64/exadata-dbmmgmt-19.2.8.0.0.191119-1.noarch.rpm /tmp/. [root@node01]# umount /mntLet's now deinstall the broken RPM:
[root@node01]# rpm -e exadata-dbmmgmt-19.2.8.0.0.191119-1.noarch --nodeps tee: /opt/oracle/dbserver_19.2.8.0.0.191119/.install_log.txt: No such file or directory 2020-11-03 15:00:05 +1100: Pre uninstallation steps in progress ... Uninstalling version 19.2.8.0.0.191119. . . . many errors as the directory has been dropped . . .And reinstall the good RPM:
[root@node01]# rpm -ivh /tmp/exadata-dbmmgmt-19.2.8.0.0.191119-1.noarch.rpm Preparing... ################################# [100%] 2020-11-03 15:00:24 +1100: Pre Installation steps in progress ... 2020-11-03 15:00:24 +1100: This is a fresh install. Updating / installing... 1:exadata-dbmmgmt-19.2.8.0.0.191119################################# [100%] 2020-11-03 15:00:34 +1100: Post Installation steps in progress ... . . . [root@node01]#In the specific case of the management package, we also need to restart the services (dbmcli -e alter dbserver restart services all) and you are good to go as if /etc/oracle/dbserver has never been dropped !
The important thing to understand here is that the patches provided in the note 888828.1 to patch your Exadatas servers are ISO containing RPMs which can then be used to reinstall some RPMs in case of something wrong happens on your systems -- and this can be very very useful !
Very nice blog, very useful content and very clear instructions. Please continue sharing you knowledge.
ReplyDeleteThanks Jose !
Delete