Some useful Kafka command

Get the max partition:offset for a topic ./kafka-run-class.sh kafka.tools.GetOffsetShell –broker-list broker-ip:port –topic topic-name Get current partition:offset for a consumer group ./kafka-consumer-groups.sh –bootstrap-server broker-ip:port –group consumer-group-name –describe Get a single record by partition:offset ./kafka-console-consumer.sh –bootstrap-server broker-ip:port –topic topic-name –max-messages 1 –partition 54 –offset 90012464 Tailing Kafka topic in real time ./kafka-console-consumer.sh –bootstrap-server broker-ip:port –topic topic-name Set a ConsumerGroup offset to latest ./kafka-consumer-groups.sh –bootstrap-server server:port… Read More »

DynamodDb access in Java(Code)

Assume we have below data in DynamoDb. All fields are String type. PK is ID, SK is name. We use DynamoDbEnhancedAsyncClient to access DynamoDb data. To get a specific record with (PK, SK): public static void testGetItem() throws Exception { CompletableFuture<Record> recordFuture = table.getItem(Key.builder() .partitionValue(“001”) .sortValue(33) .build()); System.out.println(recordFuture.get()); } To query only on PK: public static… Read More »

DynamoDB access in Java(structure)

1. dynamodb vs dynamodb-enhanced 1). software.amazon.awssdk, dynamodb 2). software.amazon.awssdk, dynamodb-enhanced, a high-level library. Can map class to DynamoDb tables. Introducing enhanced DynamoDB client in the AWS SDK for Java v2 2. Client vs Table DynamoDbClient/DynamoDbEnhancedClient is like databases access. Think you know the username/password to the database. DynamoDbTable/DynamoDbAsyncTable is the table you will operate on.… Read More »

dependencyManagement as jar

Without specifying <packaging>xxx</packaging> in pom, the default is jar. For project1, it is deployed to nexus. Then project2 can uses it as normal dependency rather than parent. project2 can still uses the version, scope inside dependencyManagement in project1. See below. project1/pom.xml <?xml version=”1.0″ encoding=”UTF-8″?> <project xmlns=”http://maven.apache.org/POM/4.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd”> <modelVersion>4.0.0</modelVersion> <groupId>com.test.pengli</groupId> <artifactId>project1</artifactId> <version>1.0-SNAPSHOT</version> <properties> <maven.compiler.source>11</maven.compiler.source> <maven.compiler.target>11</maven.compiler.target>… Read More »

dependencyManagement vs dependencies

dependencies uses the dependencies. dependencyManagement doesn’t include dependencies in project. It is mainly defines dependencies and its version, scope. Sub project specifies parent to it. When sub project defines dependencies, it won’t need to specify version, scope. project1/pom.xml This pom.xml has <packaging>pom</packaging> property. In this case, it can be used as parent in sub project. <?xml version=”1.0″… Read More »

Design Elevators

电梯调度 停车场 Elevator max people capacity max weight airstairs[only to living area], goods stair[to storage area] Building Each elevator can each level or certain levels? Inside building the button controls all elevator or just one elevator. Scheduler same direction > static > different direction when elevator is running, can we press the button for different… Read More »

Producer, Consumer with ReentrantLock, Condition

public class PC { public static void main(String[] args) { ReentrantLock lock = new ReentrantLock(); Condition added = lock.newCondition(); Condition removed = lock.newCondition(); int[] resource = new int[1]; Consumer c1 = new Consumer(lock, added, removed, resource); Producer p1 = new Producer(lock, added, removed, resource); Producer p2 = new Producer(lock, added, removed, resource); new Thread(c1).start(); new… Read More »

Token Bucket

public class TokenBucket { private long maxBucketSize; private long refillRate; // per second private long currentBucketSize; private long lastRefillTimestamp; synchronized public boolean allowRequest(int token) { refill(); if (currentBucketSize >= token) { currentBucketSize -= token; return true; } return false; } private void refill() { long now = System.nanoTime(); double add = (now – lastRefillTimestamp) /… Read More »

groupon design

rule: { rule_id, type: discount/one-time value: 30%/50$ limited_user: [usr1, usr2] limited_user_grp: [grp1, grp2] number_of_usage: 1, service: [LYFT, CHIPOTLE] } groupon: { g_id, code: XMASCODE rule_id: xxx expire_date: 2021/12/31 } usage: { user_id, groupon_id, time }