Performance Testing Report - Solution

Owned by Baseer Khan
Mar 15, 2023
5 min read

Environment Setup

Component Name

Version Number

Component Name

Version Number

Device Engine

SCE 2.0

AStack Version 2.2.0.REL22Build109

DeviceEngine Build#72

Modules Build # 70

Domain Modules

Light (DOF)

Parking (DOF)

Physical Deployment

scperf3 : t2xlarge AWS

 

scperf2 : t2xlarge AWS with local Postgres

Infrastructure Details

Virtual Machine

DE Instance

DE Configuration

Service

Virtual Machine

DE Instance

DE Configuration

Service

VM 1

CPU- 4 core

RAM- 16GB

OS- Redhat 7.3

Host Name- scperf2.tql.io

JDK Version- Amazon Coretto JDK 1.8 update 202

  1. NB/System

  2. SB

  1. Heap Size- 5GB Port- 8081

  2. Heap Size- 5GB Port- 8082

  1. Queries

  2. Updates

VM 2

CPU- 4 core

RAM- 16GB

OS- Redhat 7.3

Host Name- scperf3.tql.io

JDK Version- Amazon Coretto JDK 1.8 update 202

  1. NB/System

  2. SB

  1. Heap Size- 5GB Port- 8087

  2. Heap Size- 5GB Port- 8080

  1. Queries

  2. Updates

Performance Test tools

Tool Name

Description

Access Details

Tool Name

Description

Access Details

Grafana

To check resource utilization graphs during the performance tests

Link-  http://scperf1.tql.io:3000

User Name- admin

Password- adminnew

JMeter

We have performed the performance tests using this tool. 



Test Cases

Load Description

 

Load Description

 

Small

Small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain

Medium

Medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain

Large

Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain

Configuration Description

DE Instance

DE Instance # 2

Postgres

Redis

Configuration Description

DE Instance

DE Instance # 2

Postgres

Redis

Config Id 1

5 GB

N.A.

Remote

Local

Config Id 2

5 GB

5 GB SB

Remote

Local

Config Id 3

5 GB

N.A.

Local

Local

Config id 4

5 GB

5 GB SB

Local

Local

Data Setup : To run these test 17k Light and 9k Parking Data were loaded.

All the tests were run with 5 or more Find queries running on NB at a minute’s interval

Results for JVM Memory Used

Config Id/Load

 

Small Load (500)

Medium Load(5000)

Large Load(25000)

Config Id/Load

 

Small Load (500)

Medium Load(5000)

Large Load(25000)

Config 1

(RDS One Node)

Peak Memory Used

~ 2.3 GB

~ 3GB

~ 4 GB

Median Memory Used

~1.5 GB

~ 2 GB

~ 2.5 GB

Config 2

(RDS Two Nodes)

Peak Memory Used

~ 2.1 GB

~ 3.1 GB

~ 4.2 GB

Median Memory Used

~ 1.75 GB

~ 2 GB

~ 3 GB

Config 3

(On Prem One Node)

Peak Memory Used

~ 2 GB

~ 3.2 GB

< 5 GB

Median Memory Used

~ 1.5 GB

~ 2 GB

~ 3.5 GB

Config 4

(On Prem Two Nodes)

Peak Memory Used

~ 2 GB

~ 3.3 GB

~ 4.7 GB

Median Memory Used

~ 1 GB

~ 2 GB

~ 3 GB

  • *In case of cluster nodes with separate NB and SB, max memory of both the nodes is taken as Max Memory used

  • Heap memory is not getting released after the tests are over, the tests which continued without a DE restart show higher memory usage

All the above tests were run for 10 hours each, some tests ran continuously after the previous test, some ran after immediately after setup of data and some ran after restart of DE, the tests which were started after re-starting DE show lower memory usage.

Summary

Summary of CKC sites by Sensor count (this is a partial list)

Smart City Region Category

Count

Smart City Region Category

Count

Small ( Sensor Activity < 500 )

29

Medium ( Sensor Activity < 5000 )

5

Large (Sensor Activity < 25000 )

2

Saving of Memory with new DE 2.0 compared to previous release :

CKC Size\Memory Used

Memory Used Old Release GB

Memory Used New Release GB

CKC Size\Memory Used

Memory Used Old Release GB

Memory Used New Release GB

Small CKC

519

145

Medium CKC

104

25

Large CKC

59

20

Total Count

682

190

Detailed Result for each Configuration

Config Id 1:

Remote Database Server with Single Node

a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain

 

 

 

b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain

 

c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain

Config Id 2:

Remote Database Server with Two Nodes

a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain

b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain

 

c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain

Config Id 3:

Local PG with Single Node

a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain

 

b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain

 

c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain

 

Config Id 4:

Local PG with Two Nodes

a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain

 

b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain

 

c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain

Â