cancel
Showing results for 
Search instead for 
Did you mean: 

Oracle RMAN backup restore using netbackup DR appliance.

Rahul_Mekkalki
Level 3

Hi,

We are using Netbackup 8.1 for our backups

 Production Site and Recovery Site. Both sites are active with Master Server (name NB-Master) installed in Production Site. There is also one Media Server (name NB-Media1) installed on Production Site and at least one Media Server (name NB-Media2) on Disaster Recovery site. It means that one NetBackup domain is splitted between two sites. All backups (included NetBackup Catalog backups) are done on Production Media Server (name NB-Media1) and all backups (included NetBackup Catalog backups) replicated to the Disaster Recovery Site via Storage Lifecycle Policy (SLP).

We have testing different scenarios:

1.First scenario restore (RMAN) oracle database using  Production Media Server (name NB-Media1) where we have taken backup and it is working fine. 

2. Second scenario (Master Server (name NB-Master) and Media Server (name NB-Media1) not available). We want to restore restore (RMAN) oracle database using  DR Media Server (name NB-Media2) where our backups get replicated using SLP. This scenario is giving error.

We are initiating restore commands from client and using NB_ORA_SERV as DR appliance.

#########################

RMAN> run
{
ALLOCATE CHANNEL ch01 DEVICE TYPE SBT_TAPE PARMS="SBT_LIBRARY=/usr/openv/netbackup/bin/libobk.so64.1, ENV=(NB_ORA_CLIENT=Client_name,NB_ORA_SERV=NB-Media2)";
restore controlfile from 'cntrl_42_1_1013551832';
 }

#############################

Please note Master server & Prodution backup appliance is not reachable. DR appliance is rechable from client where we are trying to restore.

hosts file and bp.conf is having details of DR appliance.

bplist command dose not list RMAN backups on which replicated on  DR appliance. So RMAN restore fails.

How we can restore oracle database from DR appliance.

Please advise.

Regards,

Rahul.

 

 

3 ACCEPTED SOLUTIONS

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Are NB-Media1 and NB-Media2 media servers under the same master server?
(SLP is duplicating images between Media1 and and Media2)

Or do you have a different Master server at the DR site?
(SLP is replicating images to Media2 that belongs to a master server at DR site)

NB_ORA_SERV variable points to the Master server where the Image catalog resides.
bplist will contact the master server to list from the Image catalog.

So, NB_ORA_SERV=NB-Media2 can only work if  NB-Media2 is a master server, and has been added to the Oracle client in SERVER entries

View solution in original post

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If the replication and import phases completed successfully, you should be able to browse and restore using master  NB-Media2.

Can you navigate to /usr/openv/netbackup/db/images on  NB-Media2? When you list the directory names, can you see the client name from production? 
If so, what do you see when you list sub-directories in the Client folder?
There should be 10-digit folders where image headers reside. 
All 'recent' images (since 13 July) should be in a folder named 1563000000.
Image headers between 1 and 13 July will be in folder 1562000000.

So, once you have verified that their are images on NB-Media2 for the client-name in the required date-range, then please have another look at search criteria for bplist. Client name must match the folder name in images folder exactly, date range must match timestamps in images folders, etc. 
Firstly run bplist on NB-Media2 with all relevant search parameters, e.g. 
bplist -C <client-name> -s <start-date> -t 4 -R /

Does that work?
If so, try the command from the client. 
If error message is received, have a look at bprd log on NB-Media2 to see how the request is received and interpreted. 

View solution in original post

9 REPLIES 9

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Are NB-Media1 and NB-Media2 media servers under the same master server?
(SLP is duplicating images between Media1 and and Media2)

Or do you have a different Master server at the DR site?
(SLP is replicating images to Media2 that belongs to a master server at DR site)

NB_ORA_SERV variable points to the Master server where the Image catalog resides.
bplist will contact the master server to list from the Image catalog.

So, NB_ORA_SERV=NB-Media2 can only work if  NB-Media2 is a master server, and has been added to the Oracle client in SERVER entries

Hi,

Are NB-Media1 and NB-Media2 media servers under the same master server?
(SLP is duplicating images between Media1 and and Media2)

-- No.

Or do you have a different Master server at the DR site?
(SLP is replicating images to Media2 that belongs to a master server at DR site)

-- Yes we are replicating backups from NB-Media1 to NB-Media2. On Production side we have Master server (NB-Master) and Media server (NB_Media1). On DR side we have NB-Media2 server which is Master and media server.

NB_ORA_SERV variable points to the Master server where the Image catalog resides.
bplist will contact the master server to list from the Image catalog.

--- We are using DR appliance NB_Media2 for bplist. We want to restore using DR appliance NB-Media2. But bplist command dose not list images.

So, NB_ORA_SERV=NB-Media2 can only work if  NB-Media2 is a master server, and has been added to the Oracle client in SERVER entries

--- We are pointing NB_ORA_SERV=NB-Media2 and it is master server at DR side.We have added NB-Media2 entries in /etc/hosts and bp.conf file. We have removed Production Master server NB-Master and Media server NB-media1 entries from /etc/hosts and bp.conf file on oracle client.

We are restoring oracle database from host1 to host 2. On DR appliance NB-Media2 do we need to create file named with host2 and add entry of host1 at location /usr/openv/netbackup/db/altnames ?

Please advise.

Reagrds,

Rahul.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

If the replication and import phases completed successfully, you should be able to browse and restore using master  NB-Media2.

Can you navigate to /usr/openv/netbackup/db/images on  NB-Media2? When you list the directory names, can you see the client name from production? 
If so, what do you see when you list sub-directories in the Client folder?
There should be 10-digit folders where image headers reside. 
All 'recent' images (since 13 July) should be in a folder named 1563000000.
Image headers between 1 and 13 July will be in folder 1562000000.

So, once you have verified that their are images on NB-Media2 for the client-name in the required date-range, then please have another look at search criteria for bplist. Client name must match the folder name in images folder exactly, date range must match timestamps in images folders, etc. 
Firstly run bplist on NB-Media2 with all relevant search parameters, e.g. 
bplist -C <client-name> -s <start-date> -t 4 -R /

Does that work?
If so, try the command from the client. 
If error message is received, have a look at bprd log on NB-Media2 to see how the request is received and interpreted. 

Hi,

Firstly run bplist on NB-Media2 with all relevant search parameters, e.g. 
bplist -C <client-name> -s <start-date> -t 4 -R /

Does that work?

--- Yes bplist listing backup images on NB-Media2 DR master media server.

If so, try the command from the client. 

Client side it is giving error :

EXIT STATUS 25: cannot connect on socket


If error message is received, have a look at bprd log on NB-Media2 to see how the request is received and interpreted. 

BPRD logs

####################################

12:14:31.305 [13987.13987] <2> vnet_pbxAcceptSocket_ex: Accepted sock[9] from 192.168.27.52:50455
12:14:31.305 [13987.13987] <2> daemon_select_and_accept_ex: pbx setsockopt SO_KEEPALIVE succeeded
12:14:31.305 [13987.13987] <2> bprd: socket fd from accept() is 9
12:14:31.306 [13987.13987] <2> listen_loop: request complete
12:14:31.307 [50244.50244] <2> daemon_proxy_proto: Preparing to do daemon protocol for (192.168.153.10:1556 <- 192.168.27.52:50455)
12:14:31.312 [50244.50244] <16> dump_proxy_info: statusmsg: A certificate entry was not found for the requested server., nbu status = 5949, severity = 2, Additional Message: The peer proxy on host (ebsstbdb1) failed to find usable certificates. Certificates may not have been successfully deployed., nbu status = 7660, severity = 2, Additional Message: [PROXY] Encountered error (CERT_PROTOCOL_SELECT_COMMON_CA_ROOT) while processing(CertProtocol)., nbu status = 4, severity = 1
12:14:31.312 [50244.50244] <16> dump_proxy_info: local_proxy_info:
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----pid: 12580
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----connection_id: {1EB6C80C-A7AA-11E9-AB0E-C359BF6218E1}:INBOUND
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----ca_roots: cddc6d9d-6508-4d61-b26a-5ba355ae7cad
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----ca_roots_excluded: UNCONSTRAINED
12:14:31.312 [50244.50244] <16> dump_proxy_info: remote_proxy_info:
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----pid: 17546
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----connection_id: {1EB1C794-A7AA-11E9-8231-00144FFB0029}:OUTBOUND
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----ca_roots: 6d09de04-692f-42b3-a703-cc2d73e4d270
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----ca_roots_excluded: UNCONSTRAINED
12:14:31.312 [50244.50244] <16> dump_proxy_info: remote_endpoint_info:
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----pid: 25933
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----local_hostname: ebsstbdb1
12:14:31.312 [50244.50244] <16> dump_proxy_info: domain_constraints_set:
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----process_hint_server_name: repnbu01
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----process_hint: cddc6d9d-6508-4d61-b26a-5ba355ae7cad
12:14:31.312 [50244.50244] <16> dump_proxy_info: ----process_hint_reason: the local peer process is a domain service, and only accepts connections from peers within primary master domain
12:14:31.312 [50244.50244] <16> vnet_proxy_protocol_from_legacy: proxy returned status: 7660 msg: {"status": 7660, "remote_proxy_info": {"ca_roots": ["6d09de04-692f-42b3-a703-cc2d73e4d270"], "ca_roots_excluded": ["UNCONSTRAINED"], "connection_id": "{1EB1C794-A7AA-11E9-8231-00144FFB0029}:OUTBOUND", "pid": 17546}, "remote_endpoint_info": {"bpcd_info": null, "i_am_daemon": false, "domain_constraints": {}, "service": "bprd", "auth_only": false, "pid": 25933, "ssa": false, "local_user": "root", "host": "repnbu01", "peer_host": "repnbu01", "peer_client": null, "local_client": "ebsstbdb1", "local_dhcp_interval": 0, "peer_dhcp_info": null, "local_hostname": "ebsstbdb1", "secure_peer": true, "resilient": false, "nbrntd_peer_addr": null, "nbrntd_sock_addr": null, "hand_back": false, "request_timeout": 310}, "cert_status": "<?xml version=\"1.0\" encoding=\"utf-8\"?><StatusMsgDoc><StatusMsg NBUStatus=\"5949\" Severity=\"2\" VXULoid=\"486\" VXULtid=\"49\"><MsgString>A&#32;certificate&#32;entry&#32;was&#32;not&#32;found&#32;for&#32;the&#32;requested&#32;server.</MsgString></StatusMsg></StatusMsgDoc>", "local_proxy_info": {"ca_roots": ["cddc6d9d-6508-4d61-b26a-5ba355ae7cad"], "ca_roots_excluded": ["UNCONSTRAINED"], "connection_id": "{1EB6C80C-A7AA-11E9-AB0E-C359BF6218E1}:INBOUND", "pid": 12580}, "domain_constraints_set": {"process_hint": "cddc6d9d-6508-4d61-b26a-5ba355ae7cad", "process_hint_reason": "the local peer process is a domain service, and only accepts connections from peers within primary master domain", "process_hint_server_name": "repnbu01"}, "statusmsgxml": "<?xml version=\"1.0\" encoding=\"utf-8\"?><StatusMsgDoc><StatusMsg NBUStatus=\"5949\" Severity=\"2\" VXULoid=\"486\" VXULtid=\"49\"><MsgString>A&#32;certificate&#32;entry&#32;was&#32;not&#32;found&#32;for&#32;the&#32;requested&#32;server.</MsgString><StatusMsg NBUStatus=\"7660\" Severity=\"2\" VXULoid=\"486\" VXULtid=\"112\"><P pos=\"0\">ebsstbdb1</P><MsgString>The&#32;peer&#32;proxy&#32;on&#32;host&#32;(ebsstbdb1)&#32;failed&#32;to&#32;find&#32;usable&#32;certificates.&#32;&#32;Certificates&#32;may&#32;not&#32;have&#32;been&#32;successfully&#32;deployed.</MsgString></StatusMsg><StatusMsg NBUStatus=\"4\" Severity=\"1\" VXULoid=\"486\" VXULtid=\"88\"><P pos=\"0\">CERT_PROTOCOL_SELECT_COMMON_CA_ROOT</P><P pos=\"1\">CertProtocol</P><MsgString>[PROXY]&#32;Encountered&#32;error&#32;(CERT_PROTOCOL_SELECT_COMMON_CA_ROOT)&#32;while&#32;processing(CertProtocol).</MsgString></StatusMsg></StatusMsg></StatusMsgDoc>", "statusmsg": "A certificate entry was not found for the requested server., nbu status = 5949, severity = 2, Additional Message: The peer proxy on host (ebsstbdb1) failed to find usable certificates. Certificates may not have been successfully deployed., nbu status = 7660, severity = 2, Additional Message: [PROXY] Encountered error (CERT_PROTOCOL_SELECT_COMMON_CA_ROOT) while processing(CertProtocol)., nbu status = 4, severity = 1"} vnet status: 7660
12:14:31.312 [50244.50244] <16> vnet_proxy_socket_swap: vnet_proxy_protocol_from_legacy() failed: 7660
12:14:31.312 [50244.50244] <16> daemon_proxy_proto: vnet_proxy_socket_swap() failed: vnet status 7660, nb status 61
12:14:31.313 [50244.50244] <4> db_error_add_to_file: secure proxy protocol failed
12:14:31.313 [50244.50244] <32> bprd: secure proxy protocol failed
12:14:31.322 [13987.13987] <2> listen_loop: do_schild = 1
12:14:31.322 [13987.13987] <2> childterm: pid=50244 exit=61, signo=0 core=no
12:14:31.322 [13987.13987] <2> schild: wait2() ECHILD

##################

Please advise.

Regards,

Rahul.

Hi,

Thanks for help. 

Followed given docs

https://vox.veritas.com/t5/NetBackup/NetBackup-Client-Token-for-second-master-server/m-p/841485

https://vox.veritas.com/t5/NetBackup/adding-8-1-certificates-to-multiple-masters-for-trust/td-p/8419...

Now bplist listing backup images and restore also started working.

RAN below commands on client:

./nbcertcmd -getCACertificate -server Second master server

./nbcertcmd -displayCACertDetail -server Second master server

./nbcertcmd -hostselfcheck -server Second master server

./nbcertcmd -getcertificate -server Second master server

Not clear with exactly which commands are required. Please can you explain in detail.

 

Thank you once again for your expertise support.

Regards,

Rahul.

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I am not an expert on NBU security certificates, so, no idea if all of those commands were necessary.

You can find explanation of nbcertcmd options in NBU Commands manual.
Online version:  https://www.veritas.com/content/support/en_US/doc/123533878-127136857-0/v123548019-127136857

Hi Rahul

The two commands that were essential are the nbcertcmd -getCACertificate - server <master2>, followed by the nbcertcmd -getCertificate -server <master2>.
A system can be a client of multiple domains. To communicate with the master server in a domain, the system requires a hostid certificate from the master server. To get this certificate it first requires to get teh CA Certificate from the master so it can trust the master (when you run the getCACertificate command it prompts whether you want to trust the certificate with a fingerprint - this can and probably should be checked against the master server to make sure it is correct). 
Once the system has a certifcate trusted by a master server it can then participate in the NetBackup domain.
Hope that helps.

David

Hi David,

Thanks for update. Now it is clear.

Regards,

Rahul.