Performance Testing Report - Solution
Environment Setup
Component Name | Version Number |
|---|---|
Device Engine | SCE 2.0 AStack Version 2.2.0.REL22Build109 DeviceEngine Build#72 Modules Build # 70 |
Domain Modules | Light (DOF) Parking (DOF) |
Physical Deployment
scperf3 : t2xlarge AWS
scperf2 : t2xlarge AWS with local Postgres
Infrastructure Details
Virtual Machine | DE Instance | DE Configuration | Service |
|---|---|---|---|
VM 1 CPU- 4 core RAM- 16GB OS- Redhat 7.3 Host Name- scperf2.tql.io JDK Version- Amazon Coretto JDK 1.8 update 202 |
|
|
|
VM 2 CPU- 4 core RAM- 16GB OS- Redhat 7.3 Host Name- scperf3.tql.io JDK Version- Amazon Coretto JDK 1.8 update 202 |
|
|
|
Performance Test tools
Tool Name | Description | Access Details |
|---|---|---|
Grafana | To check resource utilization graphs during the performance tests | Link- http://scperf1.tql.io:3000 User Name- admin Password- adminnew |
JMeter | We have performed the performance tests using this tool. |
Test Cases
Load Description |
|
|---|---|
Small | Small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain |
Medium | Medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain |
Large | Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain |
Configuration Description | DE Instance | DE Instance # 2 | Postgres | Redis |
|---|---|---|---|---|
Config Id 1 | 5 GB | N.A. | Remote | Local |
Config Id 2 | 5 GB | 5 GB SB | Remote | Local |
Config Id 3 | 5 GB | N.A. | Local | Local |
Config id 4 | 5 GB | 5 GB SB | Local | Local |
Data Setup : To run these test 17k Light and 9k Parking Data were loaded.
All the tests were run with 5 or more Find queries running on NB at a minute’s interval
Results for JVM Memory Used
Config Id/Load |
| Small Load (500) | Medium Load(5000) | Large Load(25000) |
|---|---|---|---|---|
Config 1 (RDS One Node) | Peak Memory Used | ~ 2.3 GB | ~ 3GB | ~ 4 GB |
Median Memory Used | ~1.5 GB | ~ 2 GB | ~ 2.5 GB | |
Config 2 (RDS Two Nodes) | Peak Memory Used | ~ 2.1 GB | ~ 3.1 GB | ~ 4.2 GB |
Median Memory Used | ~ 1.75 GB | ~ 2 GB | ~ 3 GB | |
Config 3 (On Prem One Node) | Peak Memory Used | ~ 2 GB | ~ 3.2 GB | < 5 GB |
Median Memory Used | ~ 1.5 GB | ~ 2 GB | ~ 3.5 GB | |
Config 4 (On Prem Two Nodes) | Peak Memory Used | ~ 2 GB | ~ 3.3 GB | ~ 4.7 GB |
Median Memory Used | ~ 1 GB | ~ 2 GB | ~ 3 GB |
*In case of cluster nodes with separate NB and SB, max memory of both the nodes is taken as Max Memory used
Heap memory is not getting released after the tests are over, the tests which continued without a DE restart show higher memory usage
All the above tests were run for 10 hours each, some tests ran continuously after the previous test, some ran after immediately after setup of data and some ran after restart of DE, the tests which were started after re-starting DE show lower memory usage.
Summary
Summary of CKC sites by Sensor count (this is a partial list)
Smart City Region Category | Count |
|---|---|
Small ( Sensor Activity < 500 ) | 29 |
Medium ( Sensor Activity < 5000 ) | 5 |
Large (Sensor Activity < 25000 ) | 2 |
Saving of Memory with new DE 2.0 compared to previous release :
CKC Size\Memory Used | Memory Used Old Release GB | Memory Used New Release GB |
|---|---|---|
Small CKC | 519 | 145 |
Medium CKC | 104 | 25 |
Large CKC | 59 | 20 |
Total Count | 682 | 190 |
Detailed Result for each Configuration
Config Id 1:
Remote Database Server with Single Node
a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain
b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain
c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain
Config Id 2:
Remote Database Server with Two Nodes
a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain
b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain
c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain
Config Id 3:
Local PG with Single Node
a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain
b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain
c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain
Config Id 4:
Local PG with Two Nodes
a) Results for small number of Updates around 500 every 5 minutes, all 500 updates on LightDomain
b) Results for medium number of Updates around 5000 every 5 minutes, all 5000 updates on LightDomain
c) Large number of Updates around 25000 every 10 minutes, 17k updates on LightDomain and 8k updates on ParkingDomain