devbox
花生_TL007
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

聊聊hikari连接池的leakDetectionThreshold

作者：花生_TL007 | 2024-05-25 13:51:30

赞

踩

leakdetectionthreshold hikari

序

本文主要研究一下hikari连接池的leakDetectionThreshold，也就是连接池泄露检测。

leakDetectionThreshold

用来设置连接被占用的超时时间，单位为毫秒，默认为0，表示禁用连接泄露检测。

如果大于0且不是单元测试，则进一步判断：(leakDetectionThreshold < SECONDS.toMillis(2) or (leakDetectionThreshold > maxLifetime && maxLifetime > 0)，会被重置为0 .

即如果要生效则必须>0，同时满足：不能小于2秒，而且当maxLifetime > 0时不能大于maxLifetime，该值默认为1800000，即30分钟。

HikariPool.getConnection

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

  1.    /**
  2.     * Get a connection from the pool, or timeout after the specified number of milliseconds.
  3.     *
  4.     * @param hardTimeout the maximum time to wait for a connection from the pool
  5.     * @return a java.sql.Connection instance
  6.     * @throws SQLException thrown if a timeout occurs trying to obtain a connection
  7.     */
  8.    public Connection getConnection(final long hardTimeout) throws SQLException
  9.    {
  10.       suspendResumeLock.acquire();
  11.       final long startTime = currentTime();
  12.  
  13.       try {
  14.          long timeout = hardTimeout;
  15.          do {
  16.             PoolEntry poolEntry = connectionBag.borrow(timeout, MILLISECONDS);
  17.             if (poolEntry == null) {
  18.                break; // We timed out... break and throw exception
  19.             }
  20.  
  21.             final long now = currentTime();
  22.             if (poolEntry.isMarkedEvicted() || (elapsedMillis(poolEntry.lastAccessed, now) > ALIVE_BYPASS_WINDOW_MS && !isConnectionAlive(poolEntry.connection))) {
  23.                closeConnection(poolEntry, poolEntry.isMarkedEvicted() ? EVICTED_CONNECTION_MESSAGE : DEAD_CONNECTION_MESSAGE);
  24.                timeout = hardTimeout - elapsedMillis(startTime);
  25.             }
  26.             else {
  27.                metricsTracker.recordBorrowStats(poolEntry, startTime);
  28.                return poolEntry.createProxyConnection(leakTaskFactory.schedule(poolEntry), now);
  29.             }
  30.          } while (timeout > 0L);
  31.  
  32.          metricsTracker.recordBorrowTimeoutStats(startTime);
  33.          throw createTimeoutException(startTime);
  34.       }
  35.       catch (InterruptedException e) {
  36.          Thread.currentThread().interrupt();
  37.          throw new SQLException(poolName + " - Interrupted during connection acquisition", e);
  38.       }
  39.       finally {
  40.          suspendResumeLock.release();
  41.       }
  42.    }
  43. 复制代码

注意，getConnection返回的时候调用了poolEntry.createProxyConnection(leakTaskFactory.schedule(poolEntry), now) 注意，这里创建代理连接的时候关联了ProxyLeakTask

leakTaskFactory

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

  1.    /**
  2.     * Construct a HikariPool with the specified configuration.
  3.     *
  4.     * @param config a HikariConfig instance
  5.     */
  6.    public HikariPool(final HikariConfig config)
  7.    {
  8.       super(config);
  9.  
  10.       this.connectionBag = new ConcurrentBag<>(this);
  11.       this.suspendResumeLock = config.isAllowPoolSuspension() ? new SuspendResumeLock() : SuspendResumeLock.FAUX_LOCK;
  12.  
  13.       this.houseKeepingExecutorService = initializeHouseKeepingExecutorService();
  14.  
  15.       checkFailFast();
  16.  
  17.       if (config.getMetricsTrackerFactory() != null) {
  18.          setMetricsTrackerFactory(config.getMetricsTrackerFactory());
  19.       }
  20.       else {
  21.          setMetricRegistry(config.getMetricRegistry());
  22.       }
  23.  
  24.       setHealthCheckRegistry(config.getHealthCheckRegistry());
  25.  
  26.       registerMBeans(this);
  27.  
  28.       ThreadFactory threadFactory = config.getThreadFactory();
  29.  
  30.       LinkedBlockingQueue<Runnable> addConnectionQueue = new LinkedBlockingQueue<>(config.getMaximumPoolSize());
  31.       this.addConnectionQueue = unmodifiableCollection(addConnectionQueue);
  32.       this.addConnectionExecutor = createThreadPoolExecutor(addConnectionQueue, poolName + " connection adder", threadFactory, new ThreadPoolExecutor.DiscardPolicy());
  33.       this.closeConnectionExecutor = createThreadPoolExecutor(config.getMaximumPoolSize(), poolName + " connection closer", threadFactory, new ThreadPoolExecutor.CallerRunsPolicy());
  34.  
  35.       this.leakTaskFactory = new ProxyLeakTaskFactory(config.getLeakDetectionThreshold(), houseKeepingExecutorService);
  36.  
  37.       this.houseKeeperTask = houseKeepingExecutorService.scheduleWithFixedDelay(new HouseKeeper(), 100L, HOUSEKEEPING_PERIOD_MS, MILLISECONDS);
  38.    }
  39.  
  40.    /**
  41.     * Create/initialize the Housekeeping service {@link ScheduledExecutorService}.  If the user specified an Executor
  42.     * to be used in the {@link HikariConfig}, then we use that.  If no Executor was specified (typical), then create
  43.     * an Executor and configure it.
  44.     *
  45.     * @return either the user specified {@link ScheduledExecutorService}, or the one we created
  46.     */
  47.    private ScheduledExecutorService initializeHouseKeepingExecutorService()
  48.    {
  49.       if (config.getScheduledExecutor() == null) {
  50.          final ThreadFactory threadFactory = Optional.ofNullable(config.getThreadFactory()).orElse(new DefaultThreadFactory(poolName + " housekeeper", true));
  51.          final ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(1, threadFactory, new ThreadPoolExecutor.DiscardPolicy());
  52.          executor.setExecuteExistingDelayedTasksAfterShutdownPolicy(false);
  53.          executor.setRemoveOnCancelPolicy(true);
  54.          return executor;
  55.       }
  56.       else {
  57.          return config.getScheduledExecutor();
  58.       }
  59.    }
  60. 复制代码

注意，这里初始化了leakTaskFactory，以及houseKeepingExecutorService

ProxyLeakTaskFactory

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/ProxyLeakTaskFactory.java

  1. /**
  2.  * A factory for {@link ProxyLeakTask} Runnables that are scheduled in the future to report leaks.
  3.  *
  4.  * @author Brett Wooldridge
  5.  * @author Andreas Brenk
  6.  */
  7. class ProxyLeakTaskFactory
  8. {
  9.    private ScheduledExecutorService executorService;
  10.    private long leakDetectionThreshold;
  11.  
  12.    ProxyLeakTaskFactory(final long leakDetectionThreshold, final ScheduledExecutorService executorService)
  13.    {
  14.       this.executorService = executorService;
  15.       this.leakDetectionThreshold = leakDetectionThreshold;
  16.    }
  17.  
  18.    ProxyLeakTask schedule(final PoolEntry poolEntry)
  19.    {
  20.       return (leakDetectionThreshold == 0) ? ProxyLeakTask.NO_LEAK : scheduleNewTask(poolEntry);
  21.    }
  22.  
  23.    void updateLeakDetectionThreshold(final long leakDetectionThreshold)
  24.    {
  25.       this.leakDetectionThreshold = leakDetectionThreshold;
  26.    }
  27.  
  28.    private ProxyLeakTask scheduleNewTask(PoolEntry poolEntry) {
  29.       ProxyLeakTask task = new ProxyLeakTask(poolEntry);
  30.       task.schedule(executorService, leakDetectionThreshold);
  31.  
  32.       return task;
  33.    }
  34. }
  35. 复制代码

注意，如果leakDetectionThreshold=0，即禁用连接泄露检测，schedule返回的是ProxyLeakTask.NO_LEAK，否则则新建一个ProxyLeakTask，在leakDetectionThreshold时间后触发

ProxyLeakTask

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/ProxyLeakTask.java

  1. /**
  2.  * A Runnable that is scheduled in the future to report leaks.  The ScheduledFuture is
  3.  * cancelled if the connection is closed before the leak time expires.
  4.  *
  5.  * @author Brett Wooldridge
  6.  */
  7. class ProxyLeakTask implements Runnable
  8. {
  9.    private static final Logger LOGGER = LoggerFactory.getLogger(ProxyLeakTask.class);
  10.    static final ProxyLeakTask NO_LEAK;
  11.  
  12.    private ScheduledFuture<?> scheduledFuture;
  13.    private String connectionName;
  14.    private Exception exception;
  15.    private String threadName; 
  16.    private boolean isLeaked;
  17.  
  18.    static
  19.    {
  20.       NO_LEAK = new ProxyLeakTask() {
  21.          @Override
  22.          void schedule(ScheduledExecutorService executorService, long leakDetectionThreshold) {}
  23.  
  24.          @Override
  25.          public void run() {}
  26.  
  27.          @Override
  28.          public void cancel() {}
  29.       };
  30.    }
  31.  
  32.    ProxyLeakTask(final PoolEntry poolEntry)
  33.    {
  34.       this.exception = new Exception("Apparent connection leak detected");
  35.       this.threadName = Thread.currentThread().getName();
  36.       this.connectionName = poolEntry.connection.toString();
  37.    }
  38.  
  39.    private ProxyLeakTask()
  40.    {
  41.    }
  42.  
  43.    void schedule(ScheduledExecutorService executorService, long leakDetectionThreshold)
  44.    {
  45.       scheduledFuture = executorService.schedule(this, leakDetectionThreshold, TimeUnit.MILLISECONDS);
  46.    }
  47.  
  48.    /** {@inheritDoc} */
  49.    @Override
  50.    public void run()
  51.    {
  52.       isLeaked = true;
  53.  
  54.       final StackTraceElement[] stackTrace = exception.getStackTrace(); 
  55.       final StackTraceElement[] trace = new StackTraceElement[stackTrace.length - 5];
  56.       System.arraycopy(stackTrace, 5, trace, 0, trace.length);
  57.  
  58.       exception.setStackTrace(trace);
  59.       LOGGER.warn("Connection leak detection triggered for {} on thread {}, stack trace follows", connectionName, threadName, exception);
  60.    }
  61.  
  62.    void cancel()
  63.    {
  64.       scheduledFuture.cancel(false);
  65.       if (isLeaked) {
  66.          LOGGER.info("Previously reported leaked connection {} on thread {} was returned to the pool (unleaked)", connectionName, threadName);
  67.       }
  68.    }
  69. }
  70. 复制代码

可以看到NO_LEAK类里头的方法都是空操作 一旦该task被触发，则抛出Exception("Apparent connection leak detected")

ProxyConnection.close

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/ProxyConnection.java

  1.    /** {@inheritDoc} */
  2.    @Override
  3.    public final void close() throws SQLException
  4.    {
  5.       // Closing statements can cause connection eviction, so this must run before the conditional below
  6.       closeStatements();
  7.  
  8.       if (delegate != ClosedConnection.CLOSED_CONNECTION) {
  9.          leakTask.cancel();
  10.  
  11.          try {
  12.             if (isCommitStateDirty && !isAutoCommit) {
  13.                delegate.rollback();
  14.                lastAccess = currentTime();
  15.                LOGGER.debug("{} - Executed rollback on connection {} due to dirty commit state on close().", poolEntry.getPoolName(), delegate);
  16.             }
  17.  
  18.             if (dirtyBits != 0) {
  19.                poolEntry.resetConnectionState(this, dirtyBits);
  20.                lastAccess = currentTime();
  21.             }
  22.  
  23.             delegate.clearWarnings();
  24.          }
  25.          catch (SQLException e) {
  26.             // when connections are aborted, exceptions are often thrown that should not reach the application
  27.             if (!poolEntry.isMarkedEvicted()) {
  28.                throw checkException(e);
  29.             }
  30.          }
  31.          finally {
  32.             delegate = ClosedConnection.CLOSED_CONNECTION;
  33.             poolEntry.recycle(lastAccess);
  34.          }
  35.       }
  36.    }
  37.  
  38.    @SuppressWarnings("EmptyTryBlock")
  39.    private synchronized void closeStatements()
  40.    {
  41.       final int size = openStatements.size();
  42.       if (size > 0) {
  43.          for (int i = 0; i < size && delegate != ClosedConnection.CLOSED_CONNECTION; i++) {
  44.             try (Statement ignored = openStatements.get(i)) {
  45.                // automatic resource cleanup
  46.             }
  47.             catch (SQLException e) {
  48.                LOGGER.warn("{} - Connection {} marked as broken because of an exception closing open statements during Connection.close()",
  49.                            poolEntry.getPoolName(), delegate);
  50.                leakTask.cancel();
  51.                poolEntry.evict("(exception closing Statements during Connection.close())");
  52.                delegate = ClosedConnection.CLOSED_CONNECTION;
  53.             }
  54.          }
  55.  
  56.          openStatements.clear();
  57.       }
  58.    }
  59.  
  60.    final SQLException checkException(SQLException sqle)
  61.    {
  62.       SQLException nse = sqle;
  63.       for (int depth = 0; delegate != ClosedConnection.CLOSED_CONNECTION && nse != null && depth < 10; depth++) {
  64.          final String sqlState = nse.getSQLState();
  65.          if (sqlState != null && sqlState.startsWith("08") || ERROR_STATES.contains(sqlState) || ERROR_CODES.contains(nse.getErrorCode())) {
  66.             // broken connection
  67.             LOGGER.warn("{} - Connection {} marked as broken because of SQLSTATE({}), ErrorCode({})",
  68.                         poolEntry.getPoolName(), delegate, sqlState, nse.getErrorCode(), nse);
  69.             leakTask.cancel();
  70.             poolEntry.evict("(connection is broken)");
  71.             delegate = ClosedConnection.CLOSED_CONNECTION;
  72.          }
  73.          else {
  74.             nse = nse.getNextException();
  75.          }
  76.       }
  77.  
  78.       return sqle;
  79.    }
  80.  
  81. 复制代码

连接有借有还，在connection的close的时候，closeStatements，checkException会调用leakTask.cancel();取消检测连接泄露的task。另外在delegate != ClosedConnection.CLOSED_CONNECTION的时候会显示调用leakTask.cancel();

HikariPool.evictConnection

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

  1.    /**
  2.     * Evict a Connection from the pool.
  3.     *
  4.     * @param connection the Connection to evict (actually a {@link ProxyConnection})
  5.     */
  6.    public void evictConnection(Connection connection)
  7.    {
  8.       ProxyConnection proxyConnection = (ProxyConnection) connection;
  9.       proxyConnection.cancelLeakTask();
  10.  
  11.       try {
  12.          softEvictConnection(proxyConnection.getPoolEntry(), "(connection evicted by user)", !connection.isClosed() /* owner */);
  13.       }
  14.       catch (SQLException e) {
  15.          // unreachable in HikariCP, but we're still forced to catch it
  16.       }
  17.    }
  18. 复制代码

如果用户手工调用evictConnection的话，也会cancelLeakTask

实例

  1. 	/**
  2. 	 * leak-detection-threshold: 5000
  3. 	 * @throws SQLException
  4. 	 * @throws InterruptedException
  5.      */
  6. 	@Test
  7. 	public void testConnLeak() throws SQLException, InterruptedException {
  8. 		Connection conn = dataSource.getConnection();
  9. 		TimeUnit.SECONDS.sleep(10);
  10. 		String sql = "select 1";
  11. 		PreparedStatement pstmt = null;
  12. 		try {
  13. 			pstmt = (PreparedStatement)conn.prepareStatement(sql);
  14. 			ResultSet rs = pstmt.executeQuery();
  15. 			int col = rs.getMetaData().getColumnCount();
  16. 			while (rs.next()) {
  17. 				for (int i = 1; i <= col; i++) {
  18. 					System.out.print(rs.getObject(i));
  19. 				}
  20. 				System.out.println("");
  21. 			}
  22. 			System.out.println("============================");
  23. 		} catch (Exception e) {
  24. 			e.printStackTrace();
  25. 		} finally {
  26. 			//close resources
  27. 			DbUtils.closeQuietly(pstmt);
  28. 			DbUtils.closeQuietly(conn);
  29. 		}
  30. 	}
  31. 复制代码

输出

  1. 2018-01-29 22:46:31.028  WARN 1454 --- [l-1 housekeeper] com.zaxxer.hikari.pool.ProxyLeakTask     : Connection leak detection triggered for org.postgresql.jdbc.PgConnection@3abd581e on thread main, stack trace follows
  2.  
  3. java.lang.Exception: Apparent connection leak detected
  4. 	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:123) ~[HikariCP-2.7.6.jar:na]
  5. 	at com.example.demo.HikariDemoApplicationTests.testConnLeak(HikariDemoApplicationTests.java:48) ~[test-classes/:na]
  6. 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71]
  7. 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71]
  8. 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71]
  9. 	at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71]
  10. 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) ~[junit-4.12.jar:4.12]
  11. 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.12.jar:4.12]
  12. 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) ~[junit-4.12.jar:4.12]
  13. 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.12.jar:4.12]
  14. 	at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:73) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  15. 	at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:83) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  16. 	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  17. 	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  18. 	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  19. 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) ~[junit-4.12.jar:4.12]
  20. 	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  21. 	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  22. 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) ~[junit-4.12.jar:4.12]
  23. 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) ~[junit-4.12.jar:4.12]
  24. 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) ~[junit-4.12.jar:4.12]
  25. 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) ~[junit-4.12.jar:4.12]
  26. 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) ~[junit-4.12.jar:4.12]
  27. 	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  28. 	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  29. 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363) ~[junit-4.12.jar:4.12]
  30. 	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) ~[spring-test-5.0.3.RELEASE.jar:5.0.3.RELEASE]
  31. 	at org.junit.runner.JUnitCore.run(JUnitCore.java:137) ~[junit-4.12.jar:4.12]
  32. 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) ~[junit-rt.jar:na]
  33. 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) ~[junit-rt.jar:na]
  34. 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) ~[junit-rt.jar:na]
  35. 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71]
  36. 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71]
  37. 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71]
  38. 	at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71]
  39. 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) ~[idea_rt.jar:na]
  40.  
  41. 1
  42. 复制代码

可以看到ProxyLeakTask抛出java.lang.Exception: Apparent connection leak detected，但是这个是在Runnable中抛出的，并不影响主线程，主线程在超时过后，仍旧继续执行，最后输出结果。

小结

hikari连接池的leakDetectionThreshold用来设置连接被占用的超时时间，单位毫秒，默认为0，即禁用连接泄露检测。这个功能相当于tomcat jdbc pool的poolCleaner里头的checkAbandoned。

不同的如下：

  • tomcat jdbc pool是采用一个timerTask，间隔timeBetweenEvictionRunsMillis时间允许一次；而hikari是每借用一个connection则会创建一个延时的定时任务，在归还或者出异常的时候cancel掉这个task
  • tomcat jdbc pool是直接abandon连接即close掉，然后该connection在后续的收发数据时会抛出异常；而hikari则是在ProxyLeakTask中主动抛出Exception("Apparent connection leak detected")。因此前者是暴力的，后者只是在Runnable抛出异常，并不影响连接的后续操作。

doc

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/花生_TL007/article/detail/622303
推荐阅读