Forum Discussion

Haniwa's avatar
Haniwa
Level 4
10 years ago
Solved

'nbdisco' causing excessive Oracle logins ?

Hi All,

The Oracle DBA is seeing Oracle logins at 6 hour intervals, throughut the day, for each Oracle SID. Since there are over 100 instances, this is messing up his log. I am suspicious of 'nbdisco' on NetBackup 7.6 client, and wondering...

1. What it's polling frequency for 'nbdisco', and if that can be changed ?

2 Will the  "REPORT_CLIENT_DISCOVERIES = FALSE" bp.conf setting disable polling for Oracle instances altogether, or just disable the reporting of instances back to the master (while continueing to poll for them) ?

3. Is there any documentation for 'nbdisco', other than its mention in "NetBackup 7.6 Oracle Admin Guide" pages 53-54 ?

-------------- Configuration --------------

NetBackup 7.6.0.3 (probably Linux based)

NetBackup 7.6.0.3 client on RHEL 6.5.

Oracle 11.2.0.3, 11.2.0.4, 12.1.0.2

____________________________

Thank you all reading.

  • Hi,

     

    If you put it to false the service/process wont run so I would say in your case that is a good idea.

     

    http://www.symantec.com/docs/HOWTO106494

     

  • From what I know to nbdisco it does not login to Oracle. It only detects running Oracle instances used by Oracle Intelligent Policies.

    Update: I did a strace on the nbdisco in-house, i did not catch any calls for Oracle binary

    20405 11:56:31 statfs("/opt/oracle/product/11gR2p1", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=2580302, f_bfree=1382353, f_bavail=1251281, f_files=1310720, f_ffree=1244722, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
    20405 11:56:31 stat("/opt/oracle/product/11gR2p1", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    20405 11:56:31 stat("/opt/oracle/product/11gR2p1", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    20405 11:56:32 read(17, "#\n# This file is used by ORACLE "..., 8191) = 789
    20405 11:56:32 open("/var/opt/oracle/oratab", O_RDONLY) = -1 ENOENT (No such file or directory
  • Hi,

     

    If you put it to false the service/process wont run so I would say in your case that is a good idea.

     

    http://www.symantec.com/docs/HOWTO106494

     

  • From what I know to nbdisco it does not login to Oracle. It only detects running Oracle instances used by Oracle Intelligent Policies.

    Update: I did a strace on the nbdisco in-house, i did not catch any calls for Oracle binary

    20405 11:56:31 statfs("/opt/oracle/product/11gR2p1", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=2580302, f_bfree=1382353, f_bavail=1251281, f_files=1310720, f_ffree=1244722, f_fsid={0, 0}, f_namelen=255, f_frsize=4096}) = 0
    20405 11:56:31 stat("/opt/oracle/product/11gR2p1", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    20405 11:56:31 stat("/opt/oracle/product/11gR2p1", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
    20405 11:56:32 read(17, "#\n# This file is used by ORACLE "..., 8191) = 789
    20405 11:56:32 open("/var/opt/oracle/oratab", O_RDONLY) = -1 ENOENT (No such file or directory
  • Thank you Riaan, Nicola. I do not have access to the Oracle servers, so results are delayed while waiting for remote engineers to carry out changes.

    Status Update:

    1. Nbdisco was disabled today (awaiting results).

              REPORT_CLIENT_DISCOVERIES = FALSE

    2. Found another setting, to change the frequency of nbdisco probes, (but did not deploy it).

             NBARS_DISCOVERY_TIMER = <seconds>

    3. From strace performed by Nicolai, it looks like 'oratab' is being read, but would be interested in knowing exactly how nbdisco discovers databases.

    4. Since unidentified database logins are reported every 6 hours (5AM, 11AM, 5PM, 11PM), I have requested a full strace of nbdisco during one of those times (awaiting results).

    5. Thinking that the database logins are made over a TCP connection (local or remote), I have also requested a 'tcpdump' on a specific database listen port, at one of those times.

    I plan to update this post when when more info is available from the remote engineers, and furthermore, I will get a 1st hand look myself during the 1st week of March.

    Thank You!

    Ken W

     

     

  • Thanks for the update Ken.

    I can add that nbdisco will find running instances of oracle even when they are removed from oratab

    My guess is that both oratab and list of running processes are investigated by nbdisco.

     

  • Nbdisco is now fully disabled, but the databases continue to get unidentified logins, once per day.

    Nbdisco is not involved, therefore further discussion is outside the scope of this forum, but I will add some info in closing:

    1. The VOM host agent is running on these hosts, and we are going to disable that as a test.

    2. The DBA has captured the Oracle login as "SYS as SYSDBA", and snippets of the SQL code being executed follow ...

    select SYS_CONTEXT('USERENV', 'SERVER_HOST'), SYS_CONTEXT('USERENV', 'DB_UNIQUE_NAME'), SYS_CONTEXT('USERENV', 'INSTANCE_NAME'), SYS_CONTEXT('USERENV', 'INSTANCE'), STARTUP_TIME, SYS_CONTEXT('USERENV', 'DB_DOMAIN'), SYS_CONTEXT('USERENV', 'SERVICE_NAME')  from v$instance

    select decode(upper(failover_method), NULL, 0 , 'BASIC', 1, 'PRECONNECT', 2 , 'PREPARSE', 4 , 0), decode(upper(failover_type), NULL, 1,  'NONE', 1, 'SESSION', 2, 'SELECT',  4, 'TRANSACTION',  8, 1), failover_retries, failover_delay, flags, nvl(replay_initiation_timeout, 300), decode(upper(session_state_consistency), 'STATIC', 16, 0)  from service$ where name = :1

    CREATE PFILE='/var/tmp/zOXQaQXsE3' FROM SPFILE='/n01/app/oracle/product/12.1.0.2/dbee/dbs/spfiledbad1.ora'

    • When I look into /var/tmp none of these files are here, so they must get deleted afterwards

    SELECT executions, end_of_fetch_count,              elapsed_time/px_servers elapsed_time,        cpu_time/px_servers     cpu_time,            buffer_gets/executions  buffer_gets   FROM (SELECT sum(executions)   as executions,                            sum(case when px_servers_executions > 0                              then px_servers_executions                                  else executions end) as px_servers,                sum(end_of_fetch_count) as end_of_fetch_count,              sum(elapsed_time) as elapsed_time,                 sum(cpu_time)     as cpu_time,                     sum(buffer_gets)  as buffer_gets            FROM   gv$sql                                      WHERE executions > 0                                 AND sql_id = :1                                    AND parsing_schema_name = :2)

    SELECT executions, end_of_fetch_count,              elapsed_time/px_servers elapsed_time,        cpu_time/px_servers     cpu_time,            buffer_gets/executions  buffer_gets   FROM (SELECT sum(executions_delta) as EXECUTIONS,                              sum(case when px_servers_execs_delta > 0                                   then px_servers_execs_delta                                       else executions_delta end) as px_servers,                sum(end_of_fetch_count_delta) as end_of_fetch_count,              sum(elapsed_time_delta) as ELAPSED_TIME,              sum(cpu_time_delta) as CPU_TIME,                      sum(buffer_gets_delta) as BUFFER_GETS          FROM   DBA_HIST_SQLSTAT s,                                   V$DATABASE d,                                         DBA_HIST_SNAPSHOT sn                           WHERE  s.dbid = d.dbid                                  AND  bitand(nvl(s.flag, 0), 1) = 0                    AND  sn.end_interval_time >                                   (select systimestamp at TIME ZONE dbtimezone                  from dual) - 7                               AND  s.sql_id = :1                                    AND  s.snap_id = sn.snap_id                           AND  s.instance_number = sn.instance_number           AND  s.dbid = sn.dbid                                 AND  parsing_schema_name = :2)

    select CDB  from v$database

     

    Thank You all !

    Ken W

     

     

     

     

     

     

  • Good tip, but that would up to the DBA as he said "I do not have access to the Oracle servers".

     

  • FINAL UPDATE & CONCLUSION:

    1. Changing password was a no-go, because any agent that can 'su oracle', can login as SYS without a password.

    2. Nbdisco and VRTSsfmh discovery was disabled using 'REPORT_CLIENT_DISCOVERIES = FALSE'  (above) and VOM Server User Guide Appendix D: https://sort.symantec.com/public/documents/vom/6.0/windowsandunix/productguides/html/User_Guide/apds01.htm

    But, unidentified Oracle logins still continue.

    3. Then, the Oracle DBA changed the Oracle Enterprise Manager host identification from physical to virtual hostname, and those logins disappeared. The DBA still does not understand why the hostname change made a difference, but does prove OEM is the cause, thus taking Symantec off the hook.

    CONCLUSION:

    Unidentifed Oracle logins originated from Oracle Enterprise Manager, not Symantec agents.

    Hope this helps someone else out and saved them a week or two of trouble shooting !

    Ken W