Tomcat killer demystified

This entry contains solution of the mystery mentioned in the previous one.

Big Detective: Dear Watson, do you know who is the killer?
Dr. Watson: No, I have no idea, it is so mysterious. I think, there has to be some bug in the JVM.
Big Detective: Interesting hypothesis. But not plausible.
Dr. Watson: But as you say "Eliminate all other factors, and the one which remains must be the truth." I do not see any other solution.
Big Detective: We will see. Let's arrange all the facts. First of all, the virtual server has only 256 megabytes of memory. What happens when it is not enough?
Dr. Watson: Well, the swap space is used.
Big Detective: Usually yes, but not in our case. Unfortunately the configuration of the virtual server does not enable swap.
Dr. Watson: I see, but what happens when the server does not have enough memory?
Big Detective: That's a good question. Linux has something called OOM killer. As you can read in Who is Who: “It is the job of the linux 'oom killer' to sacrifice one or more processes in order to free up memory for the system when all else fails.”
Dr Watson: So OOM killer kills Tomcat when Linux does not have enough memory! But it means that adding more heap to the Tomcat JVM makes the trouble even worse!
Big Detective: Exactly,Watson.

12 Responses to “Tomcat killer demystified”

  1. filemon Says:

    Cool story. 🙂 So the solution is to get rid of Tomcat, right? 😛

  2. Lukáš Křečan Says:

    The solution is to set Tomcat heap size to 64M and to reconfigure Apache and mysql to consume less memory.

  3. Abdelrahman Says:

    hey lukas, nice articles, i love the way u made the murder 😉

  4. Younes Says:

    Hello,

    i have the same problem but with JBoss AS. I have tried to install JBoss AS in a VPS with 10 GB RAM. When starting JBoss AS, it crashes with the following Error:

    -------------------------------------------------------------------------------------------------------------------
    Java HotSpot(TM) Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated
    ./run.sh: line 255: 30256 Killed "$JAVA" $JAVA_OPTS -Djava.endorsed.dirs="$JBOSS_ENDORSED_DIRS" -classpath "$JBOSS_CLASSPATH" org.jboss.Main "$@"
    -----------------------------------------------------------------------------------------------------------------------------

    any help please?

    Thanks

    Younes

  5. Lukáš Křečan Says:

    It might be the same problem. You have to check available memory in the OS. If you are approching the limit, OOM killer might be killing your process. On the other hand, 10GB is a lot, so it's likely to be caused by something else.

  6. Younes Says:

    Thank you Lukáš,

    is ther any method to disable the OOM Killer or to kill other process but not JBoss AS process? How do you resolve the problem with Tomcat?

    --regards

    younes

  7. Lukáš Křečan Says:

    I am not a Linux expert, so try Google. My advice would be to give less memory to the JBoss instance.

  8. Chuck Says:

    It sounds like you are running a VPS on GoDaddy, I am having the same issues. Can you post your optimizations for apache and mysql?

  9. Lukáš Křečan Says:

    Sorry it was hosted somewhere else and I did not have much optimization there.

  10. kongqz Says:

    I have the same issues now .I am running a VDS on GoDaddy.
    who have the solution?
    email to ,thanks a lot

  11. Configuring Jira | Anna Ljungberg Says:

    [...] Looking more closely at the error messages displayed in the catalina logs especially the following line: dispatching signal SIGTERM to handler- the VM may need to be forcibly terminated and further googling made me stumble on this very useful post. [...]

  12. Matthias Says:

    Thanks for this hint,

    I found the confirmation in the file /var/log/messages

    (On CENTOS OS)