...| 慎用 Java 8 ConcurrentHashMap 的 computeIfAbsent
发布网友
发布时间：2024-09-25 17:58
共1个回答
热心网友
时间：2024-09-29 21:21
我们先看一段代码，代码中使用 Map 的时候，有可能会这么写：
Map<String,?Value>?map;//?...Value?result?=?map.get(key);if?(null?==?result)?{????result?=?this.calculateValue(key);????map.put(key,?result);}return?result;Java 8 的 java.util.Map 里面有个方法 computeIfAbsent，能够简化以上代码：
Map<String,?Value>?map;//?...return?map.computeIfAbsent(key,?this::calculateValue);以上这种写法除了简洁，如果使用的是 java.util.concurrent.ConcurrentHashMap，还能够在并发调用的情况下确保 calculateValue 方法不会被重复调用，保证原子性。
不过，前段时间对 Apache ShardingSphere-Proxy 做压测时遇到一个问题，当 BenchmarkSQL 连接 ShardingSphere Proxy 的 Terminal 数量比较高时，其中一条很简单的插入 SQL 执行延迟增加了很多。借助 Async Profiler 发现 Java 8 ConcurrentHashMap 的 computeIfAbsent 在性能上有坑。
不了解 Apache ShardingSphere 的读者可以参考 https://github.com/apache/shardingsphere。
排查考虑到当时的压测的现象是 BenchmarkSQL 并发数（Terminals）越高，New Order 业务中一条简单且重复执行的 insert SQL 执行延时越长。但是 ShardingSphere-Proxy 的所在机器的 CPU 也没有压满，考虑是不是 Proxy 代码层面存在瓶颈，于是借助 async-profiler 对压测状态下的 Proxy JVM 采样。
./profiler.sh?-e?lock?--lock?1ms?-d?180?-o?jfr?-f?output.jfr?$PID关于 async-profiler 可以参考 https://github.com/jvm-profiling-tools/async-profiler，后续我也考虑写一些相关文章。
使用 IDEA 读取采样获得的 jfr 文件，看到 Java Monitor Blocked 事件居然有三百多万次！根据堆栈，找到 ShardingSphere 这段使用了 computeIfAbsent 代码，以下为节选：
????//?...????private?static?final?Map<String,?SQLExecutionUnitBuilder>?TYPE_TO_BUILDER_MAP?=?new?ConcurrentHashMap<>(8,?1);????//?...????public?DriverExecutionPrepareEngine(final?String?type,?final?int?maxConnectionsSizePerQuery,?final?ExecutorDriverManager<C,??,??>?executorDriverManager,?????????????????????????????????????????final?StorageResourceOption?option,?final?Collection<ShardingSphereRule>?rules)?{????????super(maxConnectionsSizePerQuery,?rules);????????this.executorDriverManager?=?executorDriverManager;????????this.option?=?option;????????sqlExecutionUnitBuilder?=?TYPE_TO_BUILDER_MAP.computeIfAbsent(type,?????????????key?->?TypedSPIRegistry.getRegisteredService(SQLExecutionUnitBuilder.class,?key,?new?Properties()));????}????//?...https://github.com/apache/shardingsphere/blob/3b840b339ac580a7247b866f5904c514b169065f/shardingsphere-infra/shardingsphere-infra-executor/src/main/java/org/apache/shardingsphere/infra/executor/sql/prepare/driver/DriverExecutionPrepareEngine.java#L65
以上这段代码在每一次 Proxy 与数据库交互前都会执行，即通过 Proxy 执行 CRUD 操作的必经之路，而且里面的 type 目前只有 2 种，分别是 JDBC.STATEMENT 和 JDBC.PREPARED_STATEMENT，所以在高并发的情况下会有大量的线程调用同一个 key 的 computeIfAbsent。
我的理解是，如果在 key 存在的情况下，computeIfAbsent 操作就不存在修改的情况了，直接 get 出来就好，那事实如何？ 看一下 computeIfAbsent 方法的实现（JDK 是 Oracle 8u311），节选代码并加了一些注释：
????public?V?computeIfAbsent(K?key,?Function<??super?K,???extends?V>?mappingFunction)?{????????if?(key?==?null?||?mappingFunction?==?null)????????????throw?new?NullPointerException();????????int?h?=?spread(key.hashCode());????????V?val?=?null;????????int?binCount?=?0;????????for?(Node<K,V>[]?tab?=?table;;)?{????????????Node<K,V>?f;?int?n,?i,?fh;????????????if?(tab?==?null?||?(n?=?tab.length)?==?0)????????????????//?Map?初始化????????????????tab?=?initTable();????????????else?if?((f?=?tabAt(tab,?i?=?(n?-?1)?&?h))?==?null)?{????????????????//?key?不存在且?hash?对应的位置还没有东西????????????????Node<K,V>?r?=?new?ReservationNode<K,V>();????????????????synchronized?(r)?{????????????????????//?初始化?hash?对应的位置，放入?kv?等操作????????????????}????????????}????????????else?if?((fh?=?f.hash)?==?MOVED)????????????????//?Map?正忙着扩容????????????????tab?=?helpTransfer(tab,?f);????????????else?{????????????????//?key?的?hash?对应的位置已经存在链表或红黑树????????????????boolean?added?=?false;????????????????synchronized?(f)?{????????????????????if?(tabAt(tab,?i)?==?f)?{????????????????????????if?(fh?>=?0)?{????????????????????????????//?去链表里面找?key????????????????????????}????????????????????????else?if?(f?instanceof?TreeBin)?{????????????????????????????//?去红黑树里面找?key????????????????????????}????????????????????}????????????????}????????????????//?省略部分代码????????????}????????}????????//?省略部分代码????????return?val;????}根据我对源码的理解，即使 key 存在，computeIfAbsent 去找 key 的时候，都会进入 synchronized 代码。 那这相比 ConcurrentHashMap 不加锁的 get 操作不就影响性能了吗？Google 一下相应的话题，发现了一些内容： https://bugs.openjdk.java.net/browse/JDK-8161372 这个问题早就有人提过了，也在 JDK 9 处理了。截至本文编写 JDK 17 已经正式发布了。
解决在目前 JDK 8 仍然盛行的环境下，我们有必要考虑如何避免上面的问题，于是相应的处理方法就诞生了：https://github.com/apache/shardingsphere/pull/13275/files
SQLExecutionUnitBuilder?result;if?(null?==?(result?=?TYPE_TO_BUILDER_MAP.get(type)))?{????result?=?TYPE_TO_BUILDER_MAP.computeIfAbsent(type,?key?->?TypedSPIRegistry.getRegisteredService(SQLExecutionUnitBuilder.class,?key,?new?Properties()));}return?result;https://github.com/apache/shardingsphere/blob/300cbe86cf3d837a925c68c9babcec4839a2078f/shardingsphere-infra/shardingsphere-infra-executor/src/main/java/org/apache/shardingsphere/infra/executor/sql/prepare/driver/DriverExecutionPrepareEngine.java#L76-L80
每次从 Map 中获取 value 前，都先用 get 做一次检查，value 不存在才使用 computeIfAbsent 放入 value。由于 ConcurrentHashMap 的 computeIfAbsent 可以保证操作原子性，这里也不需要自己加 synchronized 或者做多重检查之类的操作。
问题解决～
附：JMH 测试测试环境测试代码package?icu.wwj.jmh.dangling;import?org.openjdk.jmh.annotations.Benchmark;import?org.openjdk.jmh.annotations.Fork;import?org.openjdk.jmh.annotations.Level;import?org.openjdk.jmh.annotations.Measurement;import?org.openjdk.jmh.annotations.Scope;import?org.openjdk.jmh.annotations.Setup;import?org.openjdk.jmh.annotations.State;import?org.openjdk.jmh.annotations.Threads;import?org.openjdk.jmh.annotations.Warmup;import?java.util.Map;import?java.util.concurrent.ConcurrentHashMap;@Fork(3)@Warmup(iterations?=?3,?time?=?5)@Measurement(iterations?=?3,?time?=?5)@Threads(16)@State(Scope.Benchmark)public?class?ConcurrentHashMapBenchmark?{????private?static?final?String?KEY?=?"key";????private?static?final?Object?VALUE?=?new?Object();????private?final?Map<String,?Object>?concurrentMap?=?new?ConcurrentHashMap<>(1,?1);????@Setup(Level.Iteration)????public?void?setup()?{????????concurrentMap.clear();????}????@Benchmark????public?Object?benchGetBeforeComputeIfAbsent()?{????????Object?result?=?concurrentMap.get(KEY);????????if?(null?==?result)?{????????????result?=?concurrentMap.computeIfAbsent(KEY,?__?->?VALUE);????????}????????return?result;????}????@Benchmark????public?Object?benchComputeIfAbsent()?{????????return?concurrentMap.computeIfAbsent(KEY,?__?->?VALUE);????}}JDK 8 测试结果#?JMH?version:?1.33#?VM?version:?JDK?1.8.0_311,?Java?HotSpot(TM)?64-Bit?Server?VM,?25.311-b11#?VM?invoker:?/usr/local/java/jdk1.8.0_311/jre/bin/java#?VM?options:?-Dvisualvm.id=172855224679674?-javaagent:/home/wuweijie/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/213.5744.223/lib/idea_rt.jar=38763:/home/wuweijie/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/213.5744.223/bin?-Dfile.encoding=UTF-8#?Blackhole?mode:?full?+?dont-inline?hint?(default,?use?-Djmh.blackhole.autoDetect=true?to?auto-detect)#?Warmup:?3?iterations,?5?s?each#?Measurement:?3?iterations,?5?s?each#?Timeout:?10?min?per?iteration#?Threads:?16?threads,?will?synchronize?iterations#?Benchmark?mode:?Throughput,?ops/time#?Benchmark:?icu.wwj.jmh.dangling.ConcurrentHashMapBenchmark.benchComputeIfAbsent#?Run?progress:?0.00%?complete,?ETA?00:03:00#?Fork:?1?of?3#?Warmup?Iteration???1:?11173878.242?ops/s#?Warmup?Iteration???2:?8471364.065?ops/s#?Warmup?Iteration???3:?8766401.960?ops/sIteration???1:?8776260.796?ops/sIteration???2:?8632907.974?ops/sIteration???3:?8557264.788?ops/s#?Run?progress:?16.67%?complete,?ETA?00:02:33#?Fork:?2?of?3#?Warmup?Iteration???1:?7757506.431?ops/s#?Warmup?Iteration???2:?8176991.807?ops/s#?Warmup?Iteration???3:?8795107.589?ops/sIteration???1:?8668883.337?ops/sIteration???2:?8866318.073?ops/sIteration???3:?8848517.540?ops/s#?Run?progress:?33.33%?complete,?ETA?00:02:02#?Fork:?3?of?3#?Warmup?Iteration???1:?8154698.571?ops/s#?Warmup?Iteration???2:?8317945.491?ops/s#?Warmup?Iteration???3:?8884286.732?ops/sIteration???1:?8912555.062?ops/sIteration???2:?8894750.001?ops/sIteration???3:?8780504.227?ops/sResult?"icu.wwj.jmh.dangling.ConcurrentHashMapBenchmark.benchComputeIfAbsent":??8770884.644?±(99.9%)?210678.797?ops/s?[Average]??(min,?avg,?max)?=?(8557264.788,?8770884.644,?8912555.062),?stdev?=?125371.573??CI?(99.9%):?[8560205.847,?8981563.442]?(assumes?normal?distribution)#?JMH?version:?1.33#?VM?version:?JDK?1.8.0_311,?Java?HotSpot(TM)?64-Bit?Server?VM,?25.311-b11#?VM?invoker:?/usr/local/java/jdk1.8.0_311/jre/bin/java#?VM?options:?-Dvisualvm.id=172855224679674?-javaagent:/home/wuweijie/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/213.5744.223/lib/idea_rt.jar=38763:/home/wuweijie/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/213.5744.223/bin?-Dfile.encoding=UTF-8#?Blackhole?mode:?full?+?dont-inline?hint?(default,?use?-Djmh.blackhole.autoDetect=true?to?auto-detect)#?Warmup:?3?iterations,?5?s?each#?Measurement:?3?iterations,?5?s?each#?Timeout:?10?min?per?iteration#?Threads:?16?threads,?will?synchronize?iterations#?Benchmark?mode:?Throughput,?ops/time#?Benchmark:?icu.wwj.jmh.dangling.ConcurrentHashMapBenchmark.benchGetBeforeComputeIfAbsent#?Run?progress:?50.00%?complete,?ETA?00:01:31#?Fork:?1?of?3#?Warmup?Iteration???1:?1881091972.510?ops/s#?Warmup?Iteration???2:?1843432746.197?ops/s#?Warmup?Iteration???3:?2353506882.860?ops/sIteration???1:?2389458285.091?ops/sIteration???2:?2391001171.657?ops/sIteration???3:?2387181602.010?ops/s#?Run?progress:?66.67%?complete,?ETA?00:01:01#?Fork:?2?of?3#?Warmup?Iteration???1:?1872514017.315?ops/s#?Warmup?Iteration???2:?1855584197.510?ops/s#?Warmup?Iteration???3:?2342392977.207?ops/sIteration???1:?2378551289.692?ops/sIteration???2:?2374081014.168?ops/sIteration???3:?2389909613.865?ops/s#?Run?progress:?83.33%?complete,?ETA?00:00:30#?Fork:?3?of?3#?Warmup?Iteration???1:?1880210774.729?ops/s#?Warmup?Iteration???2:?1804266170.900?ops/s#?Warmup?Iteration???3:?2337740394.373?ops/sIteration???1:?2363741084.192?ops/sIteration???2:?2372565304.724?ops/sIteration???3:?2388015878.515?ops/sResult?"icu.wwj.jmh.dangling.C