我做的应用是以Spring为系统的基础框架,mysql为后台数据库.在tomcat上发布后,总是不能进行热部署(reload),多次reload后,就会出OutOfMemory PermGen, 为此烦恼了很久,总于下定决心找找根源. 经过3天的不懈努力,小有成果,记录下来
实际上下面的分析都已经没什么用了,如果你使用tomcat6.0.26及以后的版本,我所说的这些情况都已经被处理了,并且比我处理的还要多很多.可以下载tomcat6.0.26的源代码 看看WebappClassLoader类的处理就成了.
通过分析工具的分析(用了YourKit,以及JDK1.6/bin下的jps/jmap/jhat),发现有下面几个方面会造成memory leak.
1.SystemClassLoader与WebappClassLoader加载的类相互引用,tomcat reload只是卸载WebappClassloader中的class,SystemClassLoader是不会卸载的(否则其他应用也停止了).但是WebappClassloader加载的类被SystemClassLoader引用的化,WebappClassloader中的相关类就不会被JVM进行垃圾收集
目前发现2种容易产生这种leak的现象.
a.在使用java.lang.ThreadLocal的时候很容易产生这种情况
b.使用jdbc驱动,而且不是在tomcat中配置的公共连接池.则java.sql.DriverManager一定会产生这种现象
ThreadLocal.set(Object),如果这个Object是WebappsClassLoader加载的,使用之后没有做ThreadLocal.set(null)或者ThreadLocal.remove(),就会产生memory leak.
由于ThreadLocal实际上操作的是java.lang.Thread类中的ThreadLocalMap,Thread类是由
SystemClassLoder加载的.而这个线程实例(main thread)在tomcat reload的时候不会销毁重建,必然就产生了SystemClassLoder中的类引用WebappsClassLoader的类.
DriverManager也是由SystemClassLoder载入的,当初始化某个JDBC驱动的时候,会向DriverManager中注册该驱动,通常是***.driver,例如com.mysql.jdbc.Driver
这个Driver是通过class.forName()加载的,通常也是加载到WebappClassLoader.这就出现了两个classLoader中的类的交叉引用.导致memory leak. 解决办法:
写一个ServletContextListener,在contextDestroyed方法中统一删除当前Thread的ThreadLocalMap中的内容.
public class ApplicationCleanListener implements ServletContextListener { public void contextInitialized(ServletContextEvent event) { }
public void contextDestroyed(ServletContextEvent event) { //处理ThreadLocal
ThreadLocalCleanUtil.clearThreadLocals();
/*
* 如果数据故驱动是通过应用服务器(tomcat etc...)中配置的<公用>连接池,这里不需要否则必须卸载Driver *
* 原因: DriverManager是System classloader加载的, Driver是webappclassloader加载的, * driver保存在DriverManager中,在reload过程中,由于system * classloader不会销毁,driverManager就一直保持着对driver的引用, * driver无法卸载,与driver关联的其他类
* ,例如DataSource,jdbcTemplate,dao,service....都无法卸载 */ try {
System.out.println(\ for (Enumeration e = DriverManager.getDrivers(); e .hasMoreElements();) {
Driver driver = (Driver) e.nextElement();
if (driver.getClass().getClassLoader() == getClass() .getClassLoader()) {
DriverManager.deregisterDriver(driver); } }
} catch (Exception e) { System.out
.println(\ + e.getMessage()); } } } /**
* 这个类根据 */
public class ThreadLocalCleanUtil { /**
* 得到当前线程组中的所有线程 description: * * @return */
private static Thread[] getThreads() {
ThreadGroup tg = Thread.currentThread().getThreadGroup();
while (tg.getParent() != null) { tg = tg.getParent(); }
int threadCountGuess = tg.activeCount() + 50; Thread[] threads = new Thread[threadCountGuess]; int threadCountActual = tg.enumerate(threads); while (threadCountActual == threadCountGuess) { threadCountGuess *= 2;
threads = new Thread[threadCountGuess]; threadCountActual = tg.enumerate(threads); }
return threads; }
public static void clearThreadLocals() { ClassLoader classloader = Thread .currentThread()
.getContextClassLoader(); Thread[] threads = getThreads(); try {
Field threadLocalsField = Thread.class .getDeclaredField(\ threadLocalsField.setAccessible(true);
Field inheritableThreadLocalsField = Thread.class .getDeclaredField(\ inheritableThreadLocalsField.setAccessible(true); Class tlmClass = Class
.forName(\ Field tableField = tlmClass.getDeclaredField(\ tableField.setAccessible(true);
for (int i = 0; i < threads.length; ++i) { if (threads[i] == null) continue;
Object threadLocalMap = threadLocalsField.get(threads[i]); clearThreadLocalMap(threadLocalMap, tableField, classloader);
threadLocalMap = inheritableThreadLocalsField.get(threads[i]); clearThreadLocalMap(threadLocalMap, tableField, classloader); }
} catch (Exception e) { e.printStackTrace(); } }
private static void clearThreadLocalMap(Object map, Field internalTableField, ClassLoader classloader) throws NoSuchMethodException, IllegalAccessException, NoSuchFieldException, InvocationTargetException { if (map != null) {
Method mapRemove = map.getClass().getDeclaredMethod(\ new Class[] { ThreadLocal.class }); mapRemove.setAccessible(true);
Object[] table = (Object[]) internalTableField.get(map); int staleEntriesCount = 0; if (table != null) {
for (int j = 0; j < table.length; ++j) { if (table[j] != null) { boolean remove = false;
Object key = ((Reference) table[j]).get(); if ((key != null)
&& (key.getClass().getClassLoader() == classloader)) { remove = true;
System.out.println(\ + key.getClass().getCanonicalName() + \ }
Field valueField = table[j] .getClass()
.getDeclaredField(\ valueField.setAccessible(true); Object value = valueField.get(table[j]); if ((value != null)
&& (value.getClass().getClassLoader() == classloader)) {
remove = true;
System.out.println(\ + value.getClass().getCanonicalName() + \ }
if (remove) { if (key == null) ++staleEntriesCount; else {
mapRemove.invoke(map, new Object[] { key }); } } } } }
if (staleEntriesCount > 0) { Method mapRemoveStale = map .getClass()
.getDeclaredMethod(\ mapRemoveStale.setAccessible(true);
mapRemoveStale.invoke(map, new Object[0]); } } } }
2.对于使用mysql JDBC驱动的:mysql JDBC驱动会启动一个Timer Thread,这个线程在reload的时候也是无法自动销毁.
因此,需要强制结束这个timer
可以在上面的ApplicationCleanListener中加入如下代码: try {
Class ConnectionImplClass = Thread .currentThread() .getContextClassLoader()
.loadClass(\ if (ConnectionImplClass != null
&& ConnectionImplClass.getClassLoader() == getClass() .getClassLoader()) {
System.out.println(\
Field f = ConnectionImplClass.getDeclaredField(\ f.setAccessible(true);
Timer timer = (Timer) f.get(null); timer.cancel(); }
} catch (java.lang.ClassNotFoundException e1) { // do nothing } catch (Exception e) { System.out
.println(\ + e.getMessage()); }
3.common-logging+log4j似乎也会导致leak,看网上有人说在ApplicationCleanListene6中加入这行代码就可以:
LogFactory.release(Thread.currentThread().getContextClassLoader());
我没试成功,懒得再找原因,直接换成了slf4j+logback,没有问题.据说slf4j+logback的性能还要更好. 后记:
tomcat-6.0.26之前的版本(我用的是tomcat-6.0.18),加入上述ApplicationCleanListener后,多次reload,不会出现outOfMemory.
但要注意,第一次启动后,reload一次,内存会增加,也就是看着还是由memory Leak,但是重复reload,内存始终保持在第一次reload的大小.似乎tomcat始终保留了双WebappClassLoader.因此,配置内存要小心些,至少要保证能够load两倍的你的所有jar包的大小(当然,是指Perm的内存大小).
测试过程中最好加上 JVM参数 -verbosegc,这样,在做GC的时候可以直观的看到class被卸载.
