I could start this blog the same way as this one as everything is supposed to be easy and at a 1 click distance in any Cloud so should it be for the databases backups in Oracle Cloud !
If is it indeed truely easy to set up automatic databases backups (if you haven't manually recreated the database -- do not do that, I did it for you already :)) in a few clicks:
. . . the troubleshooting is another story . . .
Indeed, you may find this kind of output in your console one day or another:
And here, you are "at 1 click distance" to . . . the documentation and you have no way to open the backup logfiles in your browser like I would have expected and the documentation shows you how to manually troubleshoot the backups (like on any non-Cloud database **but** with the Cloud software system -- let's look at how this work then !). I can now confirm that vi, grep and friends have not been killed by The Cloud :)
First of all, you need to check the database(s) which is(are) running there. You are indeed supposed to already know it but double checking with dbcli ensure that your system is healthy from a Cloud configuration perspective and that you could go further:
In this logfile, you'll find all the RMAN logs of the backup including the RMAN code which is used by the automatic backups:
Let's explore a bit more here and have a look at a successful backup:
So where can I find the logfiles of the backups (successful or not) ?
Note that you can also directly check / grep the /opt/oracle/dcs/log/dcs-agent.log file which contains some JSON outputs of some backups as well as few kind of DEBUG information. I say that it contains some JSON outputs as in the below example, we can see information about the October 3rd failed backup, about the October 9th successful backup but nothing in between but DEBUG information:
Indeed:
Thanks for reading and enjoy the Oracle Cloud (OCI) Automatic Backups Troubleshooting !
If is it indeed truely easy to set up automatic databases backups (if you haven't manually recreated the database -- do not do that, I did it for you already :)) in a few clicks:
. . . the troubleshooting is another story . . .
Indeed, you may find this kind of output in your console one day or another:
And here, you are "at 1 click distance" to . . . the documentation and you have no way to open the backup logfiles in your browser like I would have expected and the documentation shows you how to manually troubleshoot the backups (like on any non-Cloud database **but** with the Cloud software system -- let's look at how this work then !). I can now confirm that vi, grep and friends have not been killed by The Cloud :)
First of all, you need to check the database(s) which is(are) running there. You are indeed supposed to already know it but double checking with dbcli ensure that your system is healthy from a Cloud configuration perspective and that you could go further:
[root@oci_server ~]# dbcli list-databases ID DB Name DB Type DB Version CDB Class Shape Storage Status DbHomeID ---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ---------------------------------------- d5b748a3-c9ca-4ad0-87b4-a6228b4a3e57 MYDB122 Si 12.2.0.1.190115 true Oltp ASM Configured 72371d3f-74ee-4925-b0c1-232b5a4c9145 [root@oci_server ~]#OK now, we can have the list of jobs which happened on this database (I tailed it as this command will list all the jobs, feel free to "grep Failure" if you wish):
[root@oci_server ~]# dbcli list-jobs | grep -i MYDB122 | tail -20 4c3860ac-8cf1-4987-9ef1-fc2e2442ce7a Create Regular-L0 Backup with TAG-DBTRegular-L01569481521878j3k for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc September 26, 2019 7:06:38 AM UTC Failure 6f604571-d73c-4c0f-b172-836628fc386d Create Regular-L0 Backup with TAG-DBTRegular-L01569567793191oB6 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc September 27, 2019 7:05:12 AM UTC Failure be730740-b599-4c9b-8bfe-50d1c3ec50ab Create Regular-L0 Backup with TAG-DBTRegular-L01569654323853e8o for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc September 28, 2019 7:06:54 AM UTC Failure ca020c2f-12d3-435d-aa78-22b3e8718d1d Create Regular-L0 Backup with TAG-DBTRegular-L0156974066756851B for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc September 29, 2019 7:20:54 AM UTC Failure 2d00f5a5-5ede-4fd6-b196-b460750e367b Create Regular-L0 Backup with TAG-DBTRegular-L01569827110982sy7 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc September 30, 2019 7:07:07 AM UTC Failure c8d910ef-1cd3-4310-9636-5bb6ba15cbb1 Create Regular-L0 Backup with TAG-DBTRegular-L01569913536096G7c for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 1, 2019 7:07:18 AM UTC Failure 5b9c15b8-8938-43ba-8e16-db25e8b94407 Create Regular-L0 Backup with TAG-DBTRegular-L01569999888129pmL for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 2, 2019 7:05:54 AM UTC Failure 010909de-bae0-4511-bda9-81d2955cfd75 Create Regular-L0 Backup with TAG-DBTRegular-L01570086276758tz0 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 3, 2019 7:21:21 AM UTC Failure 5aae3578-7451-4c7c-9071-c643e7f01417 Create Regular-L0 Backup with TAG-DBTRegular-L01570172676836JSm for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 4, 2019 7:05:40 AM UTC Success ea9c92b4-299e-4a15-93aa-c3f40f290a9b Delete Backup for Database name: MYDB122_phx1bf October 4, 2019 7:18:26 AM UTC Success 76d0531a-3599-41ba-b440-4261db78c292 Create Regular-L1 Backup with TAG-DBTRegular-L11570259048404mCf for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 5, 2019 7:05:49 AM UTC Success 4055c4bc-9594-491f-b18c-82c15081cf5c Delete Backup for Database name: MYDB122_phx1bf October 5, 2019 7:54:06 AM UTC Success 18424327-f775-40e5-bbdb-f73e03453c0a Create Regular-L0 Backup with TAG-DBTRegular-L01570345462290fs3 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 6, 2019 7:05:28 AM UTC Success 5a05e4c7-d857-46f0-9030-5f33f6722860 Delete Backup for Database name: MYDB122_phx1bf October 6, 2019 7:49:22 AM UTC Success a7a51004-2da4-452f-a02e-1b6f11afc87a Create Regular-L1 Backup with TAG-DBTRegular-L11570431880603Lmh for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 7, 2019 7:15:23 AM UTC Success f0404baf-84cd-4c58-a4cd-5ccbf3bd3ccd Delete Backup for Database name: MYDB122_phx1bf October 7, 2019 8:18:46 AM UTC Success 249b81d9-1efd-4494-bf6f-0bfae8acaf76 Create Regular-L1 Backup with TAG-DBTRegular-L11570518241132W91 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 8, 2019 7:05:23 AM UTC Success 2b019e8a-a466-4084-87d3-03984c037754 Delete Backup for Database name: MYDB122_phx1bf October 8, 2019 8:19:44 AM UTC Success 93303906-4af4-499e-98b9-067010e05134 Create Regular-L1 Backup with TAG-DBTRegular-L11570604660275VZv for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc October 9, 2019 7:05:56 AM UTC Success 2f04d01b-1912-43cb-91b0-2dd231013030 Delete Backup for Database name: MYDB122_phx1bf October 9, 2019 8:00:09 AM UTC Success [root@oci_server ~]#We can see here that some backups are Sucessful and some are not as we could see on the console. Let's check the latest one which failed:
[root@oci_server ~]# dbcli describe-job -i 010909de-bae0-4511-bda9-81d2955cfd75 -j { "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "status" : "Failure", "message" : "DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : oci_server: /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log.", "reports" : [ { "taskId" : "TaskZLockWrapper_7698", "taskName" : "task:TaskZLockWrapper_7698", "taskResult" : "DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : oci_server: /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log.", "startTime" : "October 03, 2019 07:21:24 AM UTC", "endTime" : "October 03, 2019 07:22:21 AM UTC", "status" : "Failure", "taskDescription" : null, "parentTaskId" : "TaskServiceRequest_7697", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Error", "updatedTime" : "October 03, 2019 07:22:21 AM UTC" }, { "taskId" : "TaskSequential_7699", "taskName" : "task:TaskSequential_7699", "taskResult" : "DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : oci_server: /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log.", "startTime" : "October 03, 2019 07:21:24 AM UTC", "endTime" : "October 03, 2019 07:22:21 AM UTC", "status" : "Failure", "taskDescription" : null, "parentTaskId" : "TaskZLockWrapper_7698", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Error", "updatedTime" : "October 03, 2019 07:22:21 AM UTC" }, { "taskId" : "TaskParallel_7700", "taskName" : "Validate backup config", "taskResult" : "", "startTime" : "October 03, 2019 07:21:24 AM UTC", "endTime" : "October 03, 2019 07:21:24 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:24 AM UTC" }, { "taskId" : "TaskZJsonRpcExt_7703", "taskName" : "Validate opc_pfile configuration", "taskResult" : "successfully validated the opc_config", "startTime" : "October 03, 2019 07:21:24 AM UTC", "endTime" : "October 03, 2019 07:21:24 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:24 AM UTC" }, { "taskId" : "TaskParallel_7705", "taskName" : "Database container validation", "taskResult" : "", "startTime" : "October 03, 2019 07:21:24 AM UTC", "endTime" : "October 03, 2019 07:21:25 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:25 AM UTC" }, { "taskId" : "TaskParallel_7708", "taskName" : "libopc existence check", "taskResult" : "", "startTime" : "October 03, 2019 07:21:25 AM UTC", "endTime" : "October 03, 2019 07:21:25 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:25 AM UTC" }, { "taskId" : "TaskZJsonRpcExt_7711", "taskName" : "Backup Validations", "taskResult" : "Successfully validated for database backup", "startTime" : "October 03, 2019 07:21:25 AM UTC", "endTime" : "October 03, 2019 07:21:46 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:46 AM UTC" }, { "taskId" : "TaskZJsonRpcExt_7713", "taskName" : "Recovery Window validation", "taskResult" : "successfully validated recovery window", "startTime" : "October 03, 2019 07:21:46 AM UTC", "endTime" : "October 03, 2019 07:21:53 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:21:53 AM UTC" }, { "taskId" : "TaskZJsonRpcExt_7716", "taskName" : "Archivelog deletion policy configuration", "taskResult" : "Skipping the Archivelog deletion policy configuration", "startTime" : "October 03, 2019 07:21:53 AM UTC", "endTime" : "October 03, 2019 07:22:01 AM UTC", "status" : "Success", "taskDescription" : null, "parentTaskId" : "TaskParallel_7715", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Info", "updatedTime" : "October 03, 2019 07:22:01 AM UTC" }, { "taskId" : "TaskZJsonRpcExt_7718", "taskName" : "Database backup", "taskResult" : "DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : oci_server: /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log.", "startTime" : "October 03, 2019 07:22:01 AM UTC", "endTime" : "October 03, 2019 07:22:21 AM UTC", "status" : "Failure", "taskDescription" : null, "parentTaskId" : "TaskSequential_7699", "jobId" : "010909de-bae0-4511-bda9-81d2955cfd75", "tags" : [ ], "reportLevel" : "Error", "updatedTime" : "October 03, 2019 07:22:21 AM UTC" } ], "createTimestamp" : "October 03, 2019 07:21:21 AM UTC", "resourceList" : [ ], "description" : "Create Regular-L0 Backup with TAG-DBTRegular-L01570086276758tz0 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc", "updatedTime" : "October 03, 2019 07:22:21 AM UTC" } [root@oci_server ~]#I am not sure why Oracle documentation points to use the "-j" option which is to have a JSON output; indeed, a regular TEXT output looks more clear to me. May be because the JSON outputs usually have more information than the TEXT outputs but for this particular case, the TEXT output has the logfile I am looking for which is enough for now:
[root@oci_server ~]# dbcli describe-job -i 010909de-bae0-4511-bda9-81d2955cfd75M Job details ---------------------------------------------------------------- ID: 010909de-bae0-4511-bda9-81d2955cfd75 Description: Create Regular-L0 Backup with TAG-DBTRegular-L01570086276758tz0 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc Status: Failure Created: October 3, 2019 7:21:21 AM UTC Message: DCS-10001:Internal error encountered: Failed to run RMAN command. Please refer log at location : oci_server: /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log. Task Name Start Time End Time Status ---------------------------------------- ----------------------------------- ----------------------------------- ---------- task:TaskZLockWrapper_7698 October 3, 2019 7:21:24 AM UTC October 3, 2019 7:22:21 AM UTC Failure task:TaskSequential_7699 October 3, 2019 7:21:24 AM UTC October 3, 2019 7:22:21 AM UTC Failure Validate backup config October 3, 2019 7:21:24 AM UTC October 3, 2019 7:21:24 AM UTC Success Validate opc_pfile configuration October 3, 2019 7:21:24 AM UTC October 3, 2019 7:21:24 AM UTC Success Database container validation October 3, 2019 7:21:24 AM UTC October 3, 2019 7:21:25 AM UTC Success libopc existence check October 3, 2019 7:21:25 AM UTC October 3, 2019 7:21:25 AM UTC Success Backup Validations October 3, 2019 7:21:25 AM UTC October 3, 2019 7:21:46 AM UTC Success Recovery Window validation October 3, 2019 7:21:46 AM UTC October 3, 2019 7:21:53 AM UTC Success Archivelog deletion policy configuration October 3, 2019 7:21:53 AM UTC October 3, 2019 7:22:01 AM UTC Success Database backup October 3, 2019 7:22:01 AM UTC October 3, 2019 7:22:21 AM UTC Failure [root@oci_server ~]#The above output makes it clear that we'll find the problem in the /opt/oracle/dcs/log/oci_server/rman/bkup/MYDB122_phx1bf/rman_backup_DBTRegular-L01570086276758tz0_2019-10-03_07-22-07-6121233765598704321.log file.
In this logfile, you'll find all the RMAN logs of the backup including the RMAN code which is used by the automatic backups:
RMAN> set echo on; 2> set command id to "010909de-bae0-4511-bda9-81d2955c"; 3> report schema; 4> show all; 5> list incarnation of database; 6> set echo on; 7> set encryption on; 8> backup force device type sbt as compressed backupset incremental level 0 database tag 'DBTRegular-L01570086276758tz0' format 'DBTRegular-L01570086276758tz0_df_%d_%I_%U_%T_%t_set%s' plus archivelog not backed up tag 'DBTRegular-L01570086276758tz0' format 'DBTRegular-L01570086276758tz0_arc_%d_%I_%U_%T_%t_set%s' ; 9> backup device type sbt as compressed backupset current controlfile tag 'DBTRegular-L01570086276758tz0' format 'DBTRegular-L01570086276758tz0_cf_%d_%I_%U_%T_%t_set%s' spfile tag 'DBTRegular-L01570086276758tz0' format 'DBTRegular-L01570086276758tz0_spf_%d_%I_%U_%T_%t_set%s' ; 10> delete force noprompt obsolete; 11> set encryption off; 12>And the error which was in this case due to a FRA full:
Starting backup at 2019/10/03 07:22:16
ORACLE error from target database:
ORA-16038: log 3 sequence# 2319 cannot be archived
ORA-19809: limit exceeded for recovery files
ORA-00312: online log 3 thread 1: '+RECO/MYDB122_PHX1BF/ONLINELOG/group_3.259.1008210757'
You can now fix your FRA full issue and the next backup should be successful !
Let's explore a bit more here and have a look at a successful backup:
[root@oci_server ~]# dbcli describe-job -i 93303906-4af4-499e-98b9-067010e05134 Job details ---------------------------------------------------------------- ID: 93303906-4af4-499e-98b9-067010e05134 Description: Create Regular-L1 Backup with TAG-DBTRegular-L11570604660275VZv for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc Status: Success Created: October 9, 2019 7:05:56 AM UTC Message: Task Name Start Time End Time Status ---------------------------------------- ----------------------------------- ----------------------------------- ---------- Validate backup config October 9, 2019 7:06:00 AM UTC October 9, 2019 7:06:00 AM UTC Success Validate opc_pfile configuration October 9, 2019 7:06:00 AM UTC October 9, 2019 7:06:00 AM UTC Success Database container validation October 9, 2019 7:06:00 AM UTC October 9, 2019 7:06:00 AM UTC Success libopc existence check October 9, 2019 7:06:00 AM UTC October 9, 2019 7:06:00 AM UTC Success Backup Validations October 9, 2019 7:06:00 AM UTC October 9, 2019 7:06:22 AM UTC Success Recovery Window validation October 9, 2019 7:06:22 AM UTC October 9, 2019 7:06:28 AM UTC Success Archivelog deletion policy configuration October 9, 2019 7:06:28 AM UTC October 9, 2019 7:06:36 AM UTC Success crosscheck database backup October 9, 2019 7:06:36 AM UTC October 9, 2019 7:06:57 AM UTC Success Database backup October 9, 2019 7:06:57 AM UTC October 9, 2019 7:59:14 AM UTC Success Restore preview validate database October 9, 2019 7:59:14 AM UTC October 9, 2019 7:59:29 AM UTC Success Backup TDE Wallet files to ObjectStore October 9, 2019 7:59:29 AM UTC October 9, 2019 7:59:29 AM UTC Success [root@oci_server ~]#Unfortunately there's no logfile information here. The JSON output neither have the logfile name when a backup is successful (I don't paste it here as it is a very long output to show ... nothing interesting at this point).
So where can I find the logfiles of the backups (successful or not) ?
- There's no logfile for the successful backups (this is not bug, this is a feature as confirmed by Oracle Support):
- The unsuccessful backup logs can be found here:
/opt/oracle/dcs/log/`hostname -s`/rman/bkup/DB_UNIQUE_NAME/rman_backup_UNIQUE_BACKUP_TAG_A_TIMESTAMP.log
Note that you can also directly check / grep the /opt/oracle/dcs/log/dcs-agent.log file which contains some JSON outputs of some backups as well as few kind of DEBUG information. I say that it contains some JSON outputs as in the below example, we can see information about the October 3rd failed backup, about the October 9th successful backup but nothing in between but DEBUG information:
"parentTaskId" : "TaskSequential_7699",
"jobId" : "010909de-bae0-4511-bda9-81d2955cfd75",
"tags" : [ ],
"reportLevel" : "Error"
} ],
"createTimestamp" : "Thu Oct 03, 2019 07:21:21.018 (UTC) [1570087281018]",
"resourceList" : [ ],
"description" : "Create Regular-L0 Backup with TAG-DBTRegular-L01570086276758tz0 for Db:MYDB122 in OSS:bYmzpVkMq1BLgxHxZSwc"
}
2019-10-09 23:39:19,456 INFO [dw-494] [] c.o.d.c.t.CustomSslContextFactory: entering SSLEngine
2019-10-09 23:39:20,571 INFO [dw-53] [] c.o.d.c.t.CustomSslContextFactory: entering SSLEngine
2019-10-09 23:39:20,602 INFO [dw-513 - GET /jobs?count=100] [] c.o.d.a.r.JobsApi: Received GET request to getJobsList on JobsApi
2019-10-09 23:39:20,602 INFO [dw-513 - GET /jobs?count=100] [] c.o.d.a.r.JobsApi: where clause:
2019-10-09 23:39:48,764 INFO [dw-72] [] c.o.d.c.t.CustomSslContextFactory: entering SSLEngine
2019-10-09 23:39:49,742 INFO [dw-53] [] c.o.d.c.t.CustomSslContextFactory: entering SSLEngine
2019-10-09 23:39:49,814 INFO [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.a.r.JobsApi: Received GET request to getJobDetail on JobsApi with jobid = 93303906-4af4-499e-98b9-067010e05134
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=93303906-4af4-499e-98b9-067010e05134 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskParallel_10120 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10123 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskParallel_10125 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskParallel_10128 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10131 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10133 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10136 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10138 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10140 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10142 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,818 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ServiceJobReport: Add TaskReport r.id=TaskZJsonRpcExt_10144 jid=93303906-4af4-499e-98b9-067010e05134 status=Success
2019-10-09 23:39:49,820 DEBUG [dw-41 - GET /jobs/93303906-4af4-499e-98b9-067010e05134] [] c.o.d.c.t.r.ReportApi: Job Report:
{
"updatedTime" : "Wed Oct 09, 2019 07:59:29.518 (UTC) [1570607969518]",
"jobId" : "93303906-4af4-499e-98b9-067010e05134",
"status" : "Success",
"message" : null,
"reports" : [ {
Indeed:
[root@oci_server ~]# grep 010909de-bae0-4511-bda9-81d2955cfd75 /opt/oracle/dcs/log/dcs-agent.log | wc -l # October 3rd failed backup 48 [root@oci_server ~]# grep 76d0531a-3599-41ba-b440-4261db78c292 /opt/oracle/dcs/log/dcs-agent.log | wc -l # October 5th successful backup 0 [root@oci_server ~]# grep a7a51004-2da4-452f-a02e-1b6f11afc87a /opt/oracle/dcs/log/dcs-agent.log | wc -l # October 7th successful backup 0 [root@oci_server ~]# grep 93303906-4af4-499e-98b9-067010e05134 /opt/oracle/dcs/log/dcs-agent.log | wc -l # October 9th successful backup 5505 [root@oci_server ~]#Oracle just says "If you require more information, review the /opt/oracle/dcs/log/dcs-agent.log file." so let's take it like this as it seems that we don't have all the information we expect in this file. By the way, it is interesting to note that this "DCS agent" was already here with ODA (Oracle Database Appliance) and is used to "allow you to manage your database through the cloud platform". It then obviously needs to be improved as we are unable to see a logfile through the console ! (which may come later on)
Thanks for reading and enjoy the Oracle Cloud (OCI) Automatic Backups Troubleshooting !
No comments:
Post a Comment