Problems with Standalone grid : Cannot start database with srvctl, Enterprise manager cannot find listener
1
vote
0
answers
519
views
First of all, thanks for your time and effort on this, I've been looking into this problem for about two weeks but still haven't figured it out yet. I know it might be due to different root causes, but I would appreciate any tips and hints as well:
I am facing more than one problem, but I suspect all of them have the same root cause. Something like a permission problem :
I am trying a demo installation in VirtualBox with Grid Infrastructure 11.2.0.1 and Oracle database 11.2.0.1.. The OS is OEL 5.11 and I have used ASM for Storage management. The directory structure is as follows (Sorry I put all of the output completely because there might be something missing and I might have overlooked it) :
Under user "grid" which is the Grid Infrastructure owner:
[grid@orcl-demo ~]$ ls -l /u01
total 24
drwxrwxr-x 6 grid oinstall 4096 Jul 31 17:49 app
drwxr-xr-x 4 oracle oinstall 4096 Aug 2 13:07 installation_source
drwxrwxr-x 2 grid oinstall 16384 Jul 30 18:33 lost+found
[grid@orcl-demo ~]$ ls -l /u01/app
total 16
drwxrwxr-x 3 grid oinstall 4096 Jul 30 21:07 11.2.0
drwxrwxr-x 6 grid oinstall 4096 Aug 2 13:19 grid
drwxrwxr-x 8 oracle oinstall 4096 Aug 2 13:31 oracle
drwxrwx--- 6 grid oinstall 4096 Aug 2 13:27 oraInventory
[grid@orcl-demo ~]$ echo $ORACLE_BASE
/u01/app/grid
[grid@orcl-demo ~]$ echo $ORACLE_HOME
/u01/app/11.2.0/grid
Under user "oracle" which is the Database owner :
[oracle@orcl-demo ~]$ echo $ORACLE_HOME
/u01/app/oracle/product/11.2.0/db_1
[oracle@orcl-demo ~]$ echo $ORACLE_BASE
/u01/app/oracle
I have used official documentation and some oracle workshop guides to do the installation. Now everything works fine, except for the following things :
srvctl start database throws the following :
[oracle@orcl-demo ~]$ srvctl start database -d oradb
PRCR-1079 : Failed to start resource ora.oradb.db
CRS-5011: Check of resource "oradb" failed: details at "(:CLSN00007:)" in "/u01/app/11.2.0/grid/log/orcl-demo/agent/ohasd/oraagent_grid/oraagent_grid.log"
CRS-2674: Start of 'ora.oradb.db' on 'orcl-demo' failed
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
But I can start the database pretty smoothly using sqlplus :
[oracle@orcl-demo ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.1.0 Production on Tue Aug 6 16:26:50 2019
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startup
ORACLE instance started.
Total System Global Area 2087780352 bytes
Fixed Size 2214936 bytes
Variable Size 1258292200 bytes
Database Buffers 822083584 bytes
Redo Buffers 5189632 bytes
Database mounted.
Database opened.
SQL>
From Enterprise manager, I cannot start or stop the database smoothly, though it stops the database, it throws an error like below in the emoms.log file and in the web, it throws an "internal error" :
2019-08-06 16:31:22,297 [EMUI_16_31_19_/console/database/instance/sitemap] WARN jdbc.ConnectionCache _getConnection.353 - Got a fatal exeption when getting a connection; Error code = 1034; Cleaning up cache and retrying
2019-08-06 16:31:22,707 [MetadataLoad https://orcl-demo.training.com:3938/emd/main:A0000000004.xml] ERROR eml.XMLLoader markFileAsError.858 - Error repeated 1 times: ERROR-400|ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
2019-08-06 16:31:23,013 [EMUI_16_31_19_/console/database/instance/sitemap] ERROR perf.sitemap getBean.3488 - java.sql.SQLException: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
java.sql.SQLException: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:77)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:472)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:414)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:406)
at oracle.jdbc.driver.T4CTTIoauthenticate.receiveOsesskey(T4CTTIoauthenticate.java:309)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:405)
at oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:621)
So I have to start database manually. But EM still shows the database is down, and I get the following error in the logs :
2019-08-06 16:35:25,948 [XMLLoader0 00000000022.xml] ERROR eml.XMLLoader LoadFiles.696 - Error while processing 00000000022.xml:
java.sql.SQLException: ORA-20101: Metric PhysAddress for target type netapp_filer not configured
ORA-06512: at "SYSMAN.ECM_CT", line 718
ORA-06512: at line 1
ORA-06512: at "SYSMAN.EMD_LOADER", line 4931
ORA-06512: at line 1
and I have to restart em to make things work.
I have tried many, many different solutions, my biggest suspicion was listener, the listener has been configured automatically during Grid installation, and I just added a static listener.ora file (not sure about the contents, though):
[grid@orcl-demo ~]$ cat /u01/app/11.2.0/grid/network/admin/listener.ora
# listener.ora Network Configuration File: /u01/app/11.2.0/grid/network/admin/listener.ora
# Generated by Oracle configuration tools.
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = orcl-demo.training.com)(PORT = 1521))
)
)
SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(GLOBAL_DBNAME = oradb.training.com)
(SID_NAME = oradb)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)))
ADR_BASE_LISTENER = /u01/app/grid
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER = ON
And the status of listener service is as follows, the UNKNOWN state has happened since adding the static file. If I remove that, all of the statuses will be READY, but I won't be able to browse listener in Enterprise Manager:
[grid@orcl-demo ~]$ lsnrctl status
LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 06-AUG-2019 16:39:23
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=orcl-demo.training.com)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
Start Date 06-AUG-2019 15:33:27
Uptime 0 days 1 hr. 5 min. 56 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Parameter File /u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File /u01/app/grid/diag/tnslsnr/orcl-demo/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=orcl-demo.training.com)(PORT=1521)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM", status READY, has 1 handler(s) for this service...
Service "oradb.training.com" has 2 instance(s).
Instance "oradb", status UNKNOWN, has 1 handler(s) for this service...
Instance "oradb", status READY, has 1 handler(s) for this service...
Service "oradbXDB.training.com" has 1 instance(s).
Instance "oradb", status READY, has 1 handler(s) for this service...
The command completed successfully
I tried the installation of both applications using the same user (oracle) but then there will be a lot of issues regarding which home (ORACLE or GRID) should be called. I used the permissions in this document (though for 12c, but it works like a charm).
I am stuck! Any Help?
Asked by aLuViAn
(195 rep)
Aug 6, 2019, 12:16 PM