This post is about WebLogic don’t start. So we are going to fix it.
I hate java and its ecosystem. Some days ago one of our WebLogic run out of disk space and the process died.
WebLogic don’t start – the problem
After we enlarged our disk, partition and filesystem, WebLogic still refuses to start with some weird message:
Connecting to t3://localhost:7001 with userid weblogic ... This Exception occurred at some date and time. javax.naming.CommunicationException: Failed to initialize JNDI context, tried 1 time or times totally, the interval of each time is 0ms.. t3://localhost:7001: Destination 127.0.0.1, 7001 unreachable.; nested exception is:. <------>java.net.ConnectException: Connection refused (Connection refused); No available router to destination.; nested exception is:. <------>java.rmi.ConnectException: No available router to destination. [Root exception is java.net.ConnectException: t3://localhost:7001: Destination 127.0.0.1, 7001 unreachable.; nested exception is:. <------>java.net.ConnectException: Connection refused (Connection refused); No available router to destination.; nested exception is:. <------>java.rmi.ConnectException: No available router to destination.] --- Error al conectar al AdminServer, intentando iniciar... --- Already connected to a Node Manager Starting server AdminServer ... This Exception occurred at some date and time. weblogic.nodemanager.NMException: Received error message from Node Manager Server: [Server start command for WebLogic server 'AdminServer' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server
For some reason, there weren’t a route to the local machine(!). At least Weblogic said so, and Weblogic don’t start. Just in case I’ve double checked /etc/hosts file to see if there is an entry for localhost, and of course there is one.
That error about no router to destination repeated over and over. Let’s check node manager log:
<INFO> <base_domain> <AdminServer> <Starting WebLogic server with command line: $DOMAIN/bin/startWebLogic.sh > <INFO> <base_domain> <AdminServer> <Working directory is '$DOMAIN'> <INFO> <base_domain> <AdminServer> <Server output log file is '$DOMAIN/servers/AdminServer/logs/AdminServer.out'> <INFO> <base_domain> <AdminServer> <The server 'AdminServer' with process id 5793 is no longer alive; waiting for the process to die.> <INFO> <base_domain> <AdminServer> <Server failed during startup. It may be retried according to the auto restart configuration.> <INFO> <base_domain> <AdminServer> <Server failed but will not be restarted because the maximum number of restart attempts has been exceeded> <WARNING> <Server start command for WebLogic server 'AdminServer' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server 'AdminServer' log for detailed information.
The only useful information on the node manager log I’ve found is the path to the AdminServer log. That admin server log is full of those really long java debug messages that almost didn’t tell anything. But in all that text something call my attention:
(...) <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: Error reading replicas property file, the file may be corrupted - original Exception: java.lang.NumberFormatException with message: null> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED.> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.> (...)
After some googling for several minutes, I’ve found the cause, the name of the file and the solution to that error: there is this file $DOMAIN/servers/AdminServer/data/ldap/conf/replicas.prop that was blank, not corrupted in my case, but blank, blank as in zero bytes size. And that’s why WebLogic didn’t start.
WebLogic don’t start – Solution
The solution? delete, or rename that damned file and try to restart WebLogic again and that file will be recreated. Or maybe I don’t hate Java (maybe), but I hate weblogic.
So, you know how to fix WebLogic don’t start.
