Sample Header Ad - 728x90

Aerospike memory usage increasing constantly with time

0 votes
1 answer
393 views
I am new to Aerospike and exploring TTL features. I have a varity of keys stored in different sets and namespaces All the keys pushed are having TTL (from 10 minutes to 24 hours) This is how sets look like -
aql> show sets
+------------------+-------------+---------+-------------------+---------------+-------------------+-------------------+--------------+------------+
| disable-eviction | ns          | objects | stop-writes-count | set           | memory_data_bytes | device_data_bytes | truncate_lut | tombstones |
+------------------+-------------+---------+-------------------+---------------+-------------------+-------------------+--------------+------------+
| "false"          | "tvpreprod" | "35255" | "0"               | "data"        | "1684670710"      | "0"               | "0"          | "0"        |
| "false"          | "tvpreprod" | "12239" | "0"               | "epg"         | "3037722712"      | "0"               | "0"          | "0"        |
| "false"          | "tvpreprod" | "6019"  | "0"               | "account"     | "6300040"         | "0"               | "0"          | "0"        |
| "false"          | "tvpreprod" | "24847" | "0"               | "epg_account" | "59688725"        | "0"               | "0"          | "0"        |
| "false"          | "tvstage"   | "2958"  | "0"               | "data"        | "66575414"        | "0"               | "0"          | "0"        |
| "false"          | "tvstage"   | "1877"  | "0"               | "account"     | "1989341"         | "0"               | "0"          | "0"        |
| "false"          | "tvstage"   | "18090" | "0"               | "epg_account" | "30313634"        | "0"               | "0"          | "0"        |
| "false"          | "tvstage"   | "5202"  | "0"               | "epg"         | "1798086251"      | "0"               | "0"          | "0"        |
+------------------+-------------+---------+-------------------+---------------+-------------------+-------------------+--------------+------------+
[xx.xx.xx.xx:3000] 8 rows in set (0.848 secs)
I am using Java Client to put and get the data from Aerospike. Code to push data to Aerospike -
public void setWithTTL(String key, String value, Integer ttl) throws AerospikeException {
        try {
            Key primaryKey = new Key(config.getNameSpace(), config.getSetName(), key);
            WritePolicy policy = new WritePolicy();
            policy.expiration = ttl;
            Bin keyBin = new Bin(config.getBinNameKey(), key);
            Bin valueBin = new Bin(config.getBinNameValue(), value);
            aerospikeClient.put(policy, primaryKey, keyBin, valueBin);
            logCacheOperation(OPERATION.PUT, primaryKey, valueBin);
        } catch (AerospikeException e) {
            log.error("AerospikeException @ PUT", e);
            throw e;
        }
    }
Code to get the data -
public String get(String key) throws AerospikeException {
        try {
            Key primaryKey = new Key(config.getNameSpace(), config.getSetName(), key);
            Record record = aerospikeClient.get(aerospikeClient.getReadPolicyDefault(), primaryKey);
            logCacheOperation(OPERATION.GET, primaryKey, record);
            return record.bins.get(config.getBinNameValue()).toString();
        } catch (AerospikeException e) {
            log.error("AerospikeException @ GET", e);
            throw e;
        }
    }
Aerospike Client is created using -
ClientPolicy clientPolicy = new ClientPolicy();
            clientPolicy.user = "test";
            clientPolicy.password = "xxxx";

        String[] nodes = "x.x.x.x:3000,y.y.y.y:3000".split(",");
        Host[] hosts = new Host[nodes.length];
        AerospikeClient aerospikeClient = new AerospikeClient(clientPolicy, hosts);
Aerospike configuration file looks like this -
# Aerospike database configuration file for use with systemd.



service {

        user root

        group root

    paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.

        pidfile /var/run/aerospike/asd.pid

        service-threads 20

    proto-fd-max 15000

        cluster-name tv-hu-prod

        node-id-interface ens5

}



logging {

        # Log file must be an absolute path.

        file /var/log/aerospike/aerospike.log {

                context any info

        }



}



network {

    service {

        address any

        access-address xx.xx.xx.xx

        port 3000

    }



    heartbeat {

        mode mesh

        address yy.yy.yy.yy

        port 3002

        mesh-seed-address-port yy.yy.yy.yy  3002

        mesh-seed-address-port yy.yy.yy.yy  3002

    mesh-seed-address-port yy.yy.yy.yy   3002

        



        # To use unicast-mesh heartbeats, remove the 3 lines above, and see

        # aerospike_mesh.conf for alternative.



        interval 150

        timeout 10

    }



    fabric {

        port 3001

    }



    info {

        port 3003

    }

}


namespace tvstage {

    replication-factor 1

    memory-size 60G

        default-ttl 86400

    storage-engine memory

       allow-ttl-without-nsup true

}
The problem is Aerospike memory consumption is constantly increasing and after every 5-7 days memory is completely consumed and node needs to be restarted. I am not able to understand why memory is being consumed when every key is having TTL and expiry is ranging from 10 minutes to 24 hours. Please let me know if any more information is needed. Aerospike Version : C-5.4.0.4
Asked by Vibhav Singh Rohilla (199 rep)
Aug 25, 2022, 01:25 PM
Last activity: Aug 31, 2022, 03:43 AM