Suggestions for generating a node ID on Kubernetes

[f4b6a3]

Suggestions for generating a node ID on Kubernetes

[ycrao]

profile not working in docker or k8s container by java process.

using status.podIP as MY_POD_IP environment variable, then recalculate node, see getInstance() function below.

package com.example.sample.common.util;

import com.github.f4b6a3.tsid.Tsid;
import com.github.f4b6a3.tsid.TsidFactory;

/**
 * Time-Sorted Unique Identifier
 * <p> ref: <a href="https://github.com/f4b6a3/tsid-creator">f4b6a3/tsid-creator</a> </p>
 *
 */
public class TsidUtil {

    /**
     * getInstance
     *
     * @return return instance
     */
    private static Tsid getInstance() {
        // get pod ip address
        // `MY_POD_IP` env defined in k8s deployment yaml
        // example: 172.16.1.9
        String myPodIp = System.getenv("MY_POD_IP");
        if (null == myPodIp) {
            // when the 'MY_POD_IP' environment variable is not set in local development
            myPodIp = "127.0.0.1";
        }
        String[] ipArr = myPodIp.split("\\.");
        int node = (Integer.parseInt(ipArr[2])*256 + Integer.parseInt(ipArr[3]))%65536;
        
        TsidFactory factory = TsidFactory.builder()
                         .withNodeBits(16)  // max: 20 (in this case 16)
                         .withNode(node)    // max: 2^nodeBits (in this case 2^16 = 65536)
                         .build();
        return factory.create();
    }


    /**
     * next long id
     * <p>example: 38352658573940766</p>
     *
     * @return long value
     */
    public static Long nextLongId() {
        return getInstance().toLong();
    }

    /**
     * next string id
     * <p>example: 01226N0693HDH -> 01226n0693hdh</p>
     * @return string value
     */
    public static String nextId() {
        return getInstance().toString().toLowerCase();
    }
}

[fabiolimace]

Very good!

There's a similar strategy which also uses a wrapper file.

Contents of env-wrapper:

#!/bin/bash
# the node number is derived from the least significant 16 bits of the k8s pod IP
export TSIDCREATOR_NODE=`hostname --ip-address | awk -F. '{ print ($3*256 + $4) }'`
export TSIDCREATOR_NODE_COUNT=65536 # 2^16 = 65536
$*

Contents of Dockerfile:

...
COPY . .
RUN mv env-wrapper /bin/.
RUN chmod 755 /bin/env-wrapper
CMD env-wrapper myapp

Other suggestions: https://stackoverflow.com/questions/34911622/dockerfile-set-env-to-result-of-command

[fabiolimace]

I don't know. I don't know anything about Kubernetes.

Is pod ordinal index in a StatefulSet workload an option?

I found some links related to this here on Github:

• Link 1
• Link 2
• Link 3

I'll highlight this question so that some passerby can answer. It will be very helpful if anyone finds and shares a solution for this.

[fabiolimace]

This blog post shows how to generate snowflakes on Kubernetes: Generating Unique 64 bit IDs with Go on Kubernetes. The POD ID is derived from the IP address provided with an environment variable MY_IP.

TSID Creator has an environment variable that can be used to provide the POD ID: TSIDCREATOR_NODE. So you don't need to write extra Java code to detect the IP address and calculate the POD ID.

The lines below set the POD ID based on the pod IP address:

# append to ~/.profile
# POD identifier: x of 65536
# note that the maximum number of IDs per second per POD is reduced 64K/second/POD
# where x is the MODULO 65536 (2^16) of the first POD's IPv4 (if there's 1 or more addresses)
# for example, if the first address of the host is 10.42.10.1, the value of x is 2561 (10*256 + 1)
export TSIDCREATOR_NODE="`hostname -I | awk '{print $1}' | awk -F. '{print ($3*256 + $4) % 65536}'`"
export TSIDCREATOR_NODE_COUNT="65536"

The example takes the default cluster_cidr into account, according to this page: https://www.suse.com/support/kb/doc/?id=000020167.

By default, the cluster_cidr block is 10.42.0.0/16 and the node-cidr-mask-size is 24. This gives the cluster 256 blocks of /24 networks to distribute out to the pool of nodes. For example, node1 will get 10.42.0.0/24, node2 will get 10.42.1.0/24, node3 will get 10.42.2.0/24 and so on.

[fabiolimace]

Hi @ycrao !

So my example using ~/.profile is useless. I'll try to fix the example another day. Thanks for letting me know.

As for your Java class, the code in getInstance() is all you need to do to derive the node ID from the pod IP, besides defining the MY_POD_IP environment variable in your k8s deployment yml file. Additionally, you can also set the node bits to 16 to accommodate all 65,536 possible values.

// ...
int node = (Integer.parseInt(ipArr[2])*256 + Integer.parseInt(ipArr[3]))%65536;
TsidFactory factory = TsidFactory.builder()
                         .withNodeBits(16)  // max: 20 (in this case 16)
                         .withNode(node)    // max: 2^nodeBits (in this case 2^16 = 65536)
                         .build();
//...

Thank you for your collaboration!

[ycrao]

@fabiolimace .profile file works in local physical/virtual machine or cloud VPS. OS will initialize some config (user profile included) when machine power-on.
In docker container, it is difficult to pass a dynamic environment variable to a main process. Maybe we can write a shell file to source profile and then run java app in backgroud.

start.sh code like below:

#!/bin/bash
source /root/.profile && java -jar /sample.jar 

dockerfile code:

FROM eclipse-temurin:8-jre
# ...
COPY .profile /root/.profile
COPY start.sh /start.sh
ADD sample.jar /sample.jar
# ENTRYPOINT exec java -jar /sample.jar
CMD ["/bin/bash", "/start.sh"]