As an important part of distributed architecture, distributed global ID generator bears the pressure to share the bottleneck of database writing in high concurrency scenarios.
Previously, PHP + Swoole version has been implemented, and the performance and stability run well in the production environment. This time Java is used for rewriting, and the current test performance is good. Let me briefly introduce the project of the Java version.
Technical Architecture:Netty + Zookeeper + Redis Protocol
- Netty：It is a NIO-based client-side, server-side programming framework (similar to swoole). Netty is used as a server application to receive client requests, encode and decode Redis protocol data, and respond to redis client requests.
- Zookeeper：There are two main roles. One is that each node of generator service needs to define a unique serverId in advance and register to zk, which manages the connection state of cluster nodes in a unified way; the other is self-increasing sequence.sequenceIdIt can be managed by zk. The self-increasing sequence in the cluster shares a value in milliseconds. However, this involves resource competition and network transmission of shared locks. The performance is very poor. It is not opened by default. Here are the specific pressure measurements.
Following is QPS data (using the default concurrency 50) that is measured locally in php, java, and native redis-server using redis-benchmark pressure measurement tools.
PHPVersion of the test command:redis-benchmark -h 127.0.0.1 -p 9501 -t get -n 200000 -q
JavaVersion of the test command:redis-benchmark -h 127.0.0.1 -p 3308 -t get -n 200000 -q
The performance of self-incremental sequence Id is worrying after it is managed by zk.redis-benchmark -h 127.0.0.1 -p 9300 -t get -n 2000 -q
Native redis-server test command:redis-benchmark -h 127.0.0.1 -p 6379 -t get -n 200000 -q
Judging from the pressure data, Java version has nearly doubled the performance improvement compared with PHP version, and has approached redis server written in C language.
Only the get command is supported, and the key format is fixed as follows:
172.19.19.233:9303> get id.1
172.19.19.233:9303> get id.1.3
172.19.19.233:9303> get parse.4612393968093626395
[1, 2018:11:16 15:15:02.707, 0, 1, 11]
get id.1 Get a single id, 1 is the business id, supporting 0-1023
get id.1.3 Get three IDS at one time, one is business ID
get parse.4612393968093626395 Parse ID information, reverse time stamp, business id, cluster node id, self-increasing ID information.
Typing other commands returns error information.
Link of this Article: Introduction of Java Distributed ID Generator Technology