Domain Creation taking lot of time or hung

DOMAIN creation is hung or taking long time…

 

Or do you see your server startup is taking ages?

Or it takes hell lot of time for password less authentication (using configuration keys created by storeUserConfig wlst command), for weblogic and nodemanager.

 

Recently while installating 12c SOA Suite for a client, we faced a similar situation where in the domain creation was stuck at the 2^nd last page (oui page no: 19 as in screen shot below) for more than 20 minutes.

By experience from the past installations it is known that the “Configuration Progress” page of domain creation using config.sh, usually gets stuck at 45%-55% of its progress. But this 20+ minutes of wait time is a killer.

 

 

Cause: Blame it on concept of computing entropy.

 

I learnt about the term “entropy”, which refers to randomness of a molecules, or something like that, in a chemistry class during my school days.

In computing world too, the concept of entropy plays significant role when it comes to generation of random numbers or bytes for various computing processes. A great example will be generation of random bytes for encryption.

 

Ø  Some Background

To check which all applications/modules on your machine make use of the entropy using /dev/random, just run grep for “/random” on your files. You’ll realize that java cryptography, weblogic environment configuration, nodemanager startup, etc. have reference to this device.

 

 

For Linux machines, this entropy is powered by the below device that which keeps generating random bytes.

$>ls /dev/random (Run cat /dev/etc in your sweet time)

 

You can list below file to view the default entropy size. This is 4096 bytes by default.

$>cat /proc/sys/kernel/random/poolsize

 

And to check how much of can your entropy still rock, you can list the below file.

$> cat /proc/sys/kernel/random/entropy_avail

A value lower than 500, worse case zero, indicates the device may be unable to shake off more randomness. Hence the dependent applications get hung.

 

 

Ø  Now the solution to our problem:

 

-          Quickie:

During time of issue, you may find that value in file “entropy_avail” is either 0 or very low. To fix this, you need to run below command as “root” user.

$ root> rngd -r /dev/urandom -o /dev/random –b

This will help you recycle the entropy, which will solve the slowness or hung state of your domain creation in a jiffy.

 

-          Permanent Fix :

Run below as root

$ root> echo "EXTRAOPTIONS=\"-r /dev/urandom\"" | cat >> /etc/sysconfig/rngd

$ root> /etc/init.d/rngd start

 

-          Caution:

The permanent solution, as you see, will replace the /dev/urandon to be the source of entropy. Now the urandom source is less secure than /dev/randon, as the former reuses the random bits, in order to ensure that it’s pool never runs out of entropy.

 

 

Ø  References:

-          Oracle Doc ID 1947394.1