cancel
Showing results for 
Search instead for 
Did you mean: 

Is GRT broken in NetBackup 8.1.1 ?

asg2ki
Level 4

Hi All,

It seems that the GRT functionality is totally broken in NetBackup 8.1.1. It used to work well on NetBackup 8.1 but since I upgraded my environment, there seems to be an internal error somewhere with the product when it comes to duplicating backup images between disk storage units and respectively populating the catalog with enttries. Debug logs and jobs are not showing anything useful apart from the following generic messages:

Error bpduplicate (pid=8228) db_IMAGE() failed: database system error (220)
Error bpduplicate (pid=8228) Status = no images were successfully processed.

This happens with both Active-Directory and Exchange based GRT images and I'm pretty confident the issue is not related to any of the third party products but NetBackup itself. While I'm still able to take properly backups from the resources and also make the initial GRT catalog activity, as I mentioned earlier the problem comes specificly when duplicating images. Most of my policies are making the duplication activities via SLP's between AdvanceDisk and MSDP pools, but for the sake of troubleshooting I did all the manual procedures on duplicating a single separate backup image between various types of DiskPools without any success. The result is all the same and the logs are not indicating any particular pointers except errors like the above one. Additionally I can see the following error entries in BPDBM log file:

bpcr_get_string_pentry: Error detected: ERROR 13

process_request: request complete: exit status 220 database system error; query type: 248

Also what I noticed is that the bpdbm.exe process is indicating crashes within the Application Event Log multiple times but not specifically when the duplication takes place:

Faulting application name: bpdbm.exe, version: 8.101.18.203, time stamp: 0x5a7683e2
Faulting module name: libdiscsrv.dll, version: 8.101.18.203, time stamp: 0x5a767eae
Exception code: 0xc0000005
Fault offset: 0x000000000000dcf8
Faulting process id: 0x18ec
Faulting application start time: 0x01d41205ce5b4590
Faulting application path: E:\Program Files\Veritas\NetBackup\bin\bpdbm.exe
Faulting module path: E:\Program Files\Veritas\NetBackup\bin\libdiscsrv.dll
Report Id: 8068aebd-6e32-4c76-8b32-a1f8738309ea
Faulting package full name:
Faulting package-relative application ID:

Any help would be appreciated.

Thanks.

17 REPLIES 17

Michal_Mikulik1
Moderator
Moderator
Partner    VIP    Accredited Certified

Hello,

probably something wrong with the installation itself.. Try Repair option.

I am using AD GRT in 8.1.1 with backup to MSDP, then with a duplication to tape, and it is running ok.

Regards

Michal

Tried that already but unfortunately it didn't make any difference.

Anyway thanks for the suggestion.

In a verbose bpdbm log, find Q_IMAGE_EXTEND_FILES_FILE. (The dot f file is the files file.). Focus on that pid. Can you find it starting nbfsd? If yes, look for query_client starting nblbc on the client. The actual IPC call is something like bpcr_getfileinfo.

If you get to nblbc on the client, get a verbose log (levels 5 and 2) and capture the ncflbc log.

Exchange and AD use the same database engine, so it's believable that they would have a common problem.

Meanwhile, I'll try it. What version of Exchange? Are you using the GUI, or bpduplicate? Are you setting an Exchange GRT proxy host in the client configuration?

Depending on what you find in the bpdbm log, I may want a bprd log.

Thanks for your input.

My Exchange version is 2016 with CU9. As for the duplication, for the moment I'm using the GUI however all my SLP's that are involved in those Exchange / AD image duplication procedures are suffering from the same situation with the same error so I'm confident that manual bpduplicate procedure would end up with the same results.

I'm not using any GRT proxy hosts since this is a single Exchange instance without DAG and I'm relying on my master + multiple media servers for all backup / duplication activities. On the other hand I noticed that my master server is present as a GRT Proxy within the BP Client settings of my Exchange server. I don't recall making the change at any point in time but in any case I can't seem to be able to remove it. Probably I'll just remove the client and reinstall it to make sure things stay on the clear line. The backup from my Exchange server is being taken straight to one of my media server's Advanced Disk pool then the image is immediately being duplicated to an MSDP pool on the very same server. I tried the same scenario with different media servers including duplication across them but there was just no change in the behavior.

As per your suggestion, I took a deep dive into the logs once again and I checked against the cross-references throughout the individual log sections. I definitely see the duplication job appear in the BPDBM log which is then referenced also in the NBFSD log on the media server where I can clearly identify the nbfsd.exe process calling the right job with the right parameters but you will be able to see that in the attached logs. The NCFLBC log is also attached so that you can see the full picture.

For your information I tested out the duplication with GRT Cataloguing turned off via the master server and in this case the image definitely gets duplicated successfully.

Also I tested out manual NFS troubleshooting mounts between the master and the media server, so we can definitely rule out any possibilities for network disruptions or lack of connectivity.

Let me know your thoughts.

Thanks for your input.
My Exchange version is 2016 with CU9. As for the duplication, for the moment I'm using the GUI however all my SLP's that are involved in those Exchange / AD image duplication procedures are suffering from the same situation with the same error so I'm confident that manual bpduplicate procedure would end up with the same results.
I'm not using any GRT proxy hosts since this is a single Exchange instance without DAG and I'm relying on my master + multiple media servers for all backup / duplication activities. On the other hand I noticed that my master server is present as a GRT Proxy within the BP Client settings of my Exchange server. I don't recall making the change at any point in time but in any case I can't seem to be able to remove it. Probably I'll just remove the client and reinstall it to make sure things stay on the clear line. The backup from my Exchange server is being taken straight to one of my media server's Advanced Disk pool then the image is immediately being duplicated to an MSDP pool on the very same server. I tried the same scenario with different media servers including duplication across them but there was just no change in the behavior.
As per your suggestion, I took a deep dive into the logs once again and I checked against the cross-references throughout the individual log sections. I definitely see the duplication job appear in the BPDBM log which is then referenced also in the NBFSD log on the media server where I can clearly identify the nbfsd.exe process calling the right job with the right parameters but you will be able to see that in the attached logs. The NCFLBC log is also attached so that you can see the full picture.
For your information I tested out the duplication with GRT Cataloguing turned off via the master server and in this case the image definitely gets duplicated successfully.
Also I tested out manual NFS troubleshooting mounts between the master and the media server, so we can definitely rule out any possibilities for network disruptions or lack of connectivity.
Let me know your thoughts.

After looking at the NCFLBC logs I found the following error, so it seems I may still have some issues with the connectivity, although I'm not sure how and why.

 

0,51216,351,351,9,1530633389474,4648,7008,0:,37:Timeout: 300 (../LBCCallback.cpp:466),23:LBCCallback::nbfs_setup,4
1,51216,309,351,815,1530633389474,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A32:INF - waiting for mount mutex...|)
1,51216,309,351,816,1530633389474,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A155:INF - running command - "E:\Program Files\Veritas\NetBackup\bin\nbfs" mount -server
nbumevm12.skynet.local -port 7394 -timeout 60  -cred 60ED58A027...|)
0,51216,311,351,10,1530633411177,4648,5472,0:,54:Sending to bpbrm:  (../BRMObserverDepreciated.cpp:374),29:BRMObserverDepreciated::write,4
0,51216,159,351,7,1530633411177,4648,5472,0:,94:Released Buffer (0x0000000002cf93e0 ) containing (1) bytes of data. (../BufferManager.cpp:504),30:BufferManager::releaseBuffer(),4
0,51216,159,351,8,1530633411177,4648,5472,0:,65:Released Buffer (0x0000000002cf93e0 ). (../BufferManager.cpp:510),30:BufferManager::releaseBuffer(),4
0,51216,311,351,11,1530633433178,4648,5472,0:,54:Sending to bpbrm:  (../BRMObserverDepreciated.cpp:374),29:BRMObserverDepreciated::write,4
0,51216,159,351,9,1530633433178,4648,5472,0:,94:Released Buffer (0x0000000002cf9020 ) containing (1) bytes of data. (../BufferManager.cpp:504),30:BufferManager::releaseBuffer(),4
0,51216,159,351,10,1530633433178,4648,5472,0:,65:Released Buffer (0x0000000002cf9020 ). (../BufferManager.cpp:510),30:BufferManager::releaseBuffer(),4
1,51216,309,351,817,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A25:INF - response - BASEDIR=|)
1,51216,309,351,818,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A54:INF - response - MNTPATH=/NBUFS_A_9E85B2E10F3F3618_009|)
1,51216,309,351,819,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A28:INF - response - MOUNTPOINT=|)
1,51216,309,351,820,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A61:INF - response - aborting connection, timeout period exceeded|)
1,51216,309,351,821,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A31:INF - response - EXIT_STATUS=41|)
1,51216,309,351,822,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),2,(28|A58:INF - looking for mount path: NBUFS_A_9E85B2E10F3F3618_009|)
1,51216,309,351,823,1530633451601,4648,7008,0:,0:,12:nbfs_mount(),0,(28|A20:ERR - command failed|)
0,51216,351,351,10,1530633451601,4648,7008,0:,50:nbfs_mount() failed: %d29 (../LBCCallback.cpp:482),23:LBCCallback::nbfs_setup,1
2,51216,351,351,11,1530633451601,4648,7008,0:,0:,0:,2,(5|)
2,51216,351,351,12,1530633451601,4648,7008,0:,0:,0:,2,(5|)

 

I already have the required NFS ports opened through my firewall on each server (in fact I didn't touch them since a very long time) and the test mounts worked properly, so there must be something else I'm missing from within the picture.

Any ideas ?

Two ideas.

Since the request got to bpdbm and it launched nblbc, you have proved that the NetBackup relationships are good. The problem appears to be within nblbc itself. You can exercise this without doing a duplication. Try browsing into an Exchange mailbox in the Backup Archive and Restore window.

As you noted, the mount timed out. Reading raw VxUL logs is not pretty, but the timestamp including milliseconds is there. The nbfs mount was launched at 1530633389 seconds in ctime. It timed out at 1530633451, which is 62 seconds elapsed time. That's a problem. You can look at the two nbfsd logs on the client and the media server. The client side nbfsd log is usually tiny; all you'll see are the mounts and unmounts. The server side could be daunting to study, but it should show you what took so long.

Advice: Either run vxlogview -i 351 on the client to make the ncflbc log readable, or execute bpdbm -ctime <timestamp> to convert the time in the raw log file. If you do the latter, chop off the milliseconds, as I did in the preceding paragraph. For example (my local time is US Central Daylight Time, UTC - 0500):

C:\Program Files\Veritas\NetBackup\bin>bpdbm -ctime 1530633389
1530633389 = 7/3/2018 10:56:29 AM

Actually I already did the browsing of the backup image via BAR and it works fine on the taken backups. I can browse them down till item levels but of course without the appropriate cataloguing procedure that is always slow as hell. I see the individual restore jobs appearing and the NetBackup console jobs are indicating status 0.

Also the NCFLBC logs that I observed were not from the Exchange client itself but from the Master Server. Frankly I have no problems taking the backups at all, it's just when the post-processing catalog needs to be made, there is something blocking it, but I have no idea what. It's really strange because that whole thing used to work well enough with NBU 8.1 and I had no major changes in my environments. I have the following ports opened between the NBU servers and the clients (in both directions because I was lazy to fine tune) - 111,7394, 10082, 10102, 13782, 13724, 1556.

Unless there was another port requirement added recently I see no reason for suspecting network connectivity issues although the error 41 within the logs felt kinda like if there is a network related thing.

Anyway I'll observe the remaining logs as per your suggestion and will see if I can figure out something from there. As a matter of fact I was constantly looking at the NBFSD log on the media server but it didn't really provide much details, so I guess my best chance would be to observe things from Exchange side. At this moment I suspect there might be something related to the latest Windows or Exchange patches but hopefully the logs would tell more about the nature of this issue.

Oh and thanks for the great advice on the logging. I really wonder why Veritas wouldn't finally release a good log viewer tool for NetBackup alongside the Logging Assistant feature. LV hasn't been updated in ages now so a new tool would certainly make NBU admin's life much easier.

Anyway I'll post the details on the logs a bit later. 

 

Ok so I figured out partially what was happening although there are still few unanswered questions. So long story short this was not due to a NetBackup error but due to not having functional Client for NFS service on the Master server. I was quite surprised to figure out this is the case, because with NetBackup 8.1 product I had Clinet for NFS service installed only over the Exchange server (Enabled and Started) as well as on the Media Servers (Disabled). Also on the Media Servers I have the Server for NFS service (Disabled), and with this configuration everything worked just fine.

Now with NBU 8.1.1 it seems that something must have changed because when I installed the Client for NFS service (Enabled and Started) everything works just as expected including all my backlogged images for both Exchange and AD. I started to get very suspicous about this whole matter when I accidentally looked throughout the various log folders on the Master server and I found that the NBFSD log was actually producing quite a lot of entries as follow:

23:15:22.436 [8636.1968] <16> _nbfs_connect: WNetAddConnection2() failed, error = 1222

...until the job timed out and respectively gave me the 220 error status code. At the same time it is worth to mention that I never use my Master server for Media related tasks thus the reason why I never installed NFS binaries there.

The documentation also doesn't mention anything about requirements for NFS in order to be able to duplicate the GRT images but maybe I'm missing some points here. I know for sure that with NBU 8.1 I successfully made the GRT duplications and that I never installed NFS services on the Master server until now.

If someone could possible provide a reasonable explanation to this behavior that would be very much appreciated.

RLeon
Moderator
Moderator
   VIP   

In one of your previous replies, you said:
"On the other hand I noticed that my master server is present as a GRT Proxy within the BP Client settings of my Exchange server. I don't recall making the change at any point in time but in any case I can't seem to be able to remove it."

Did you mean the "Exchange granular proxy host" field in the Exchange client's properties has the Master Server name? If so, then that explains the need for Client for NFS on the Master Server.

In the Nbu for Exchange admin guide it has this line:
"Configuring the Exchange granular proxy host - When you browse for or restore individual items using Granular Recovery Technology (GRT), NetBackup uses the destination client to stage a virtual copy of the database that you want to restore. However, NetBackup uses the source client of the backup to stage the database in the following situations: when you duplicate or browse a backup (with bplist) that uses GRT. Alternatively, you can specify a different Windows system to act as a proxy for the source client."

That explains why it worked when you manually browse GRT items in the restore interface, because it is the Exchange client itself that initiates the Client for NFS connection to the Media Server. The Exchange client being the destination client of the restore job in this case.
It also explains why it didn't work during duplication, because your Master Server is in your client properties' Exchange proxy host field, therefore, during duplication jobs, it is the Master Server that initiates the Client for NFS connection to the Media Server. And without Client for NFS installed, the connection would fail.
If you had left this field blank, then during duplication jobs, it would have used the source client, which is also that same Exchange client - to initiate the Client for NFS connection, and it would have worked.

Your best option is to either leave that field blank, so that it would use the source Exchange client during duplication (default behavior as explained by the document excerpt above);
or, put in the name of the source Exchange client itself, which would make it do the same, but explicitly.

Thanks for the detailed explanation.

I thought of that myself too during the troubleshooting, but then it doesn't explain why things started to work suddenly also for the Active Directory image duplication as well. In the client properties of the Domain Controllers there is definitely no option for setting GRT Proxy and on my Exchange server I'm still not sure how did the Master server appear in there since I have no need to GRT Proxy functionality. In any case it was my understanding that the NFS connection is required only between the Client and the Media server for the initial collection of TRV information during the backup itself and later on for the duplication but right now it feels like if NBU master server always wants to get invloved directly into the procedure. Furthermore yesterday when things started to work again, I noticed that when my backlogged images were being processed, the NBFSD connectivity was explicitly done between the Master and the Media servers. I don't think any of the involved clients were participating actively in those connections.

The fact that my AD image duplication also started to work properly out of nowhere makes me feel like if NBU 8.1.1 has been introducing some changes to the GRT part of the image duplication.

Btw does anyone know how can I remove the hostname from the GRT field in the Exchange client properties ? I already tried clearing the field and even modifying it to the exchange client itself but it always gets back to the master server's hostname after hitting the OK button. Is there a particular registry key, file entry or anything else that I may need to modify or shall I try to reinstall the client binaries on my Exchange server ?

Ok this is now really confusing but here are the latest events.

I took the opportunity to remove the Client for NFS service from the Master server and apparently the expected behavior is back to normal at least for the Active Directory jobs. They are now duplicating images properly as expected without any 220 error status code. This is a direct proof that there is really no need for the NFS services on the Master server in normal circumstances so I can now scratch my previous assumption that something may have changed in NBU 8.1.1. Regardless I'm still very confused why on earth my backlogged images started to work only after I installed the Client for NFS. Not that there should be any relationship between the two but my AD and Exchange policies and their corresponding images are using completely separate SLP's so technically there shouldn't be any chance for having somthing mixed up.

Anyway I can confirm that the Exchange situation though was definitely due to the GRT proxy setting. I observed very carefully the traffic between the individual servers and clients and I definitely have my Master server acting on behalf of Exchange while the Domain Controllers are still communicating directly with the Media servers. It still remains a myster to me how the Exchange client got it's GRT proxy setting on, but my bigger problem with it is that I can't remove it.

I already tried reinstalling the NBU Client from Exchange (including registry and remaining file cleanup) but that didn't help at all. This makes me think that the GRT Proxy setting is handled somewhere else and I suspect that would be somewhere within the Master server properties.

Does anybody have an idea how to clear the GRT Proxy setting manually ?

Never mind the GRT Proxy, I figured it out.

It seems that the GUI method is having some sort of a bug so it just doesn't accept the changes made there. Instead one can use the "bpclient" command with the "-granular_proxy" parameter to update the properties of the particular client. After I made the changes via CLI, the GUI reflected the changes immediately and after making a small test backup I can definitely see the change in behavior although now I'm facing a different status code during the duplication of Exchange backup images:

Error bpduplicate (pid=11008) db_IMAGE() failed: client hostname could not be found (48)
Error bpduplicate (pid=11008) Status = no images were successfully processed.

I made some tests with the "bpclntcmd" command and there is absolutely no problem with the IP / naming resolution back and forth. Just in case I also cleared the cache via "bpclntcmd -clear_host_cache" but the result is all the same.

This new error happens only during image duplication and only with Exchange images. I'm still able to take backups and collect the initial TRV information but the GRT duplication just doesn't like my exchange server for some reason.

I'll take a look at the logs further but in case you have any suggestions, feel free to let me know.

I'm still not sure what exactly is failingregarding this new Error 48 but in the meantime I managed to check that the Exchange client and the Media server are definitely being able to communicate properly via NBFSD. I'm able to browse the backup image granuarly from within Backup Archive Restore client and there are no errors in the NCFLBC log files.

So the issue seems to be related somewhere else within the duplication job. I noticed that the job fails almost immediately with Error 48 so I took a look at the BPDBM log file on the Master server for more details and found the following


11:54:42.509 [10544.6512] <8> vnet_connect_to_bpcd: [vnet_connect.c:565] connect_to_service() failed 6 0x6
11:54:42.509 [10544.6512] <16> local_bpcr_connect: vnet_connect_to_bpcd() failed: 6
11:54:42.509 [10544.6512] <2> local_bpcr_connect: Can't connect to client
11:54:42.509 [10544.6512] <2> ConnectToBPCD: bpcd_connect_and_verify(, ) failed: 48
11:54:42.509 [10544.6512] <4> db_error_add_to_file: Could not connect to  for virtual browse operation, errno=0, bpcd_status=48
11:54:42.509 [10544.6512] <16> dbm_open_client_connection: Could not connect to  for virtual browse operation, errno=0, bpcd_status=48
11:54:42.509 [10544.6512] <2> dbm_close_client_connection: sending disconnect request to bpcd
11:54:42.509 [10544.6512] <2> put_short: (10) network write() error: An operation was attempted on something that is not a socket. ; socket = -1
11:54:42.509 [10544.6512] <2> bpcr_disconnect_rqst: bpcr protocol error - couldn't send request type
11:54:42.509 [10544.6512] <2> dbm_cleanup_client_connection: Entering
11:54:42.509 [10544.6512] <4> db_error_add_to_file: error while browsing from client = 48
11:54:42.509 [10544.6512] <16> extend_list_add_files: error while browsing from client = 48
11:54:42.509 [10544.6512] <2> extend_list_add_files: total_selected 3

As far as I can see from the job, it fails without even trying to communicate much with either the Media server or the Exchange client, so right now I'm stuck at looking for more clues.

Any idea which other logs I should look at for more clues ?

I got behind this discussion as I'm on leave. (Function names and other details below are approximate, from memory,)

1. In the latest post, why is there a blank for the target name here:

>Could not connect to  for virtual browse operation...

Look for the target name earlier in this bpdbm pid. (I assume this pid is the one that processes Q_IMAGE_EXTEND_FILES_FILE. Correct? You should find a log from a function something like "start_nbfsd_and_blah". Then you should find a function with query_client in its name, logging all its parameters including the client name. It then calls bpcr_get_fileinfo. This is what connects to the host where nblbc is to run.

Upstream from bpdbm, bpduplicate logs to the admin log.

2. Before you upgraded, you probably did not have the granular proxy host configuration. I don't know how it would have gotten set to the master server. Please check that when you used the CLI to clear it, you didn't accidentally set it to a space or such.

In the GUI, the granular proxy host on the Exchange tab and on the SharePoint tab are one and the same. Try clearing it on the SharePoint tab. In either case, if it's set it corresponds to a file on the master server under NetBackup\db\client\<hostname>\. This file has a name of the form g_<proxyname>. It's a touch file. If it exists, delete it.

3. The "client" is where nblbc runs. As in doc quoted earlier, it has to have NFS client enabled on it.

3. I can't explain your AD experience. I haven't spent any time with duplicating AD backup images.

1. I'm not sure but this is definitely strange. I didn't even notice this annomaly about the blank target name until you mentioned it but when I look at the Master server's BPDBM log (attached to this reply) I can certainly see the correct image ID appearing and the "start_nbfsd" function is indicating that the process is being invoked on the correct Media server. The NBFSD log on the Media server is also indicating the correct client name, backup image and job id. I found the "query_client" function in the BPDBM log and it seems to include the correct client name within the row, but on the contrary I couldn't find anything related to the "bpcr_get_fileinfo". I looked also at the ADMIN log but I couldn't find anything relevant. Just in case I attached the log sections outlining one particular manual image duplication event.

2. I never used GRT Proxy because I don't really need it in my environment and I know for sure that before the upgrade all the Exchange duplications were working well. Not sure either how the master server appeared there but I definitely managed to clear it out via the "bpclient" command. Also I looked at the path you mentioned and I couldn't find any "g_<proxyname>" files there, but just in case I made a manual modification to the client via the GUI and then the file appeared. When I tried again to change or remove it via GUI, the newly created file just stayed there (potential bug ???), but when I deleted the file manually, the GUI immediately indicated a blank field. BTW it is good to know that this is where the proxy information is being stored.

3. The NFS client was always on and enabled on my Exchange server and when I make a full backup I can see the NFS daemons doing their jobs, so I suppose the NFS section is intact in all places right now. As for the AD part, while I'm also confused about the behavior, I wouldn't worrie right now about it because it works.

 

In any case right now the only problem I've got is with the duplication of Exchange images and only when the Exchange client is trying to work on its own without the help of any GRT proxies. I already can confirm that when I used the Master server as a GRT proxy, the duplications work fine, so I guess we can narrow down the circle of possibilites a bit.

Ok the situation is as follow.

After not finding anything relevant within the logs I started to think a bit out of the ordinary considering the fact that the GRT duplication is working well via GRT Proxy and it doesn't if it is cleared. Also the missing "target" within the "Could not connect to  for virtual browse operation" message kept bugging me quite a lot. So I took the opportunity and I set the GRT Proxy to the Exchange client itself and now everything seems to be working fine. I'm not sure what the default GRT Proxy value is set to with new clients but clearing any previously set value doesn't seem to revert GRT Proxy settings back to its defaults.

Once my image got successfully duplicated, I made another test by taking a brand new full backup of Exchange but this time by clearing the GRT Proxy settings once again. The duplication failed miserably with the same "client hostname could not be found (48)" error so I guess that the default GRT Proxy must be some sort of an invisible variable pointing to the client itself. After this failure I once again set the GRT Proxy to be the client's hostname and the duplication went through smoothly without any issues.

So it seems that although totally unrelated to the initial query, we found two bugs in NBU which should be fixed.

1. The GUI can't change / clear the GRT Proxy settings in a client once it has been set.

2. The GRT Proxy doesn't revert to it's default value after being cleared manually (either via bpclient command or by removing the corresponding G_proxyname touch file).

 

Well... the good news is that my environment now works just as expected, so now all backup images are taken properly and GRT duplicated without the need of the direct NFS involvement from the Master server. My assumptions on the potential NBU 8.1.1 changes were totally wrong and apparently NBU 8.1.1 works just fine with GRT. The only thing I could think of as a root cause for all these troubles is that perhaps something may have gone wrong during the last upgrade that I made on my NBU environment. I consider this matter as an error (yet unanswered though) on my end rather than any issues with the NBU product except for the already mentioned bugs we just found.

Special thanks to Lowell_Palecek for the LOG guidance as well as to RLeon for the GRT Proxy pointers. Your help definitely saved me tons of time and most definitely helped me to resolve the issue.

Cheers