| Table of Contents | ||||||
|---|---|---|---|---|---|---|
|
Cluster Terminology
This topic introduces some of the concepts unique to the cluster creation and communication feature of A-Stack.
| Terminology | Description | Synonym Terms |
|---|---|---|
Network | A collection of A-Stack running instances that communicates (notifications) and keep persistent data store in sync. | Cluster, Clustering |
| Peer | A-Stack running instance that is participating in forming a Network | |
| Node | A-Stack running instance | Peer if participating in forming a network. |
| Group | Logical Name defined to make communication type between a collection of Nodes | |
| Advertise | Send Notifications to peer or group | |
| Replicate | Replication of data (TQL Storage) between nodes or a group of nodes |
...
For the Network Definiton below: It results in following variable values.
| Name | Value |
|---|---|
| QName | AMQPCluster.Region1_Cluster1_Server1 |
| GroupName | Region1_Cluster |
| RoutingKey | cluster.Region1_Cluster1 (Note that RoutingKey is unique per Exchange / Cluster definiton) |
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
<NewPackage>
<RuntimeParams>
<RegionName>Region1</RegionName>
<RegionSectionName>Cluster1</RegionSectionName>
<ClusterName>[:RuntimeParams.RegionName:]_[:RuntimeParams.RegionSectionName:]</ClusterName>
</RuntimeParams>
<NewFacetInstance Name="wsNotificationCluster" Type="SffNetworkFacet" Context="new" fid="wsNotificationCluster">
<OnActivate>
<ImportFacet>[:RuntimeParams.MacroFacetID:]</ImportFacet>
<SetLocalData Key="peers">
<Value>
<Server Id="[:RuntimeParams.ClusterName:]_Server1" Host="bkhan" Port="8082"/>
<Server Id="[:RuntimeParams.ClusterName:]_Server2" Host="bkhan" Port="8083"/>
<Server Id="[:RuntimeParams.ClusterName:]_Server3" Host="bkhan" Port="8084"/>
<Server Id="[:RuntimeParams.ClusterName:]_Server4" Host="bkhan" Port="8085"/>
</Value>
</SetLocalData>
<For Each="server" From="$LocalData.peers" In="Server">
<AddProcessData Key="ClusterPeers.Peer">
<Value>
<Name>[:$LocalData.server.Id:]</Name>
<URL>ws://[:$LocalData.server.Host:]:[:$LocalData.server.Port:]/fid-wsNotificationCluster</URL>
<AdvertiseTo>[:RuntimeParams.ClusterName:]</AdvertiseTo>
<Group>[:RuntimeParams.ClusterName:]</Group>
</Value>
</AddProcessData>
</For>
<Log Level="INFO" Message="TQLNotificationCluster : Federation AdvertiseTo servers [:$ProcessData.DeviceEngines:]"/>
<Process>
<Network>
<OnPeerError>
<Log Level="error" Message="TQLNotificationCluster : Peer [:$ContextData.$Peer.Key:] error: [:$Error:]"/>
</OnPeerError>
<Namespace Name="AMQPCluster">
<Include>$ProcessData.ClusterPeers</Include>
</Namespace>
<Group name="[:RuntimeParams.ClusterName:]" UserName="tql" Password="tql12345"
VirtualHost="/" Durability="true"
Host="mqtt.atomiton.com" Port="5672"
ExchangeName="AtomitonFanoutExchange"
ExchangeType="fanout" QueueName="peer.[:PeerQName:]">
<Send post="amqp://?ClientType=publisher&
UserName=[:UserName:]&
Password=[:Password:]&
VirtualHost=[:VirtualHost:]&
HostName=[:Host:]&
PortNumber=[:Port:]&
ExchangeName=[:ExchangeName:]&
ExchangeType=[:ExchangeType:]&
QueueName=[:QueueName:]&
Durability=[:Durability:]">
<RoutingKey>cluster.[:QName:]</RoutingKey>
<Template>
<Message>
<Value>
<PublishMessage>[:$Request.Message.Value:]</PublishMessage>
</Value>
</Message>
</Template>
</Send>
<Receive get="amqp://?ClientType=subscriber&
UserName=[:UserName:]&
Password=[:Password:]&
VirtualHost=[:VirtualHost:]&
HostName=[:Host:]&
PortNumber=[:Port:]&
ExchangeName=[:ExchangeName:]&
ExchangeType=[:ExchangeType:]&
QueueName=[:QueueName:]&
Durability=[:Durability:]"
as="ServerPipeline" disable="CMD_SEND">
<Message>
<Value>
<AMQPSubscriptionConfig>
<RoutingKey value="cluster.[:QName:]"/>
</AMQPSubscriptionConfig>
</Value>
</Message>
</Receive>
</Group>
</Network>
</Process>
<SffReport Name="wsNotificationCluster_OnActivate"/>
</OnActivate>
<OnOpen ModifyPipeline="WsServerExtensionArgs" Context="keep"/>
<Policy name="MyMetering" type="SffSimplePolicy" kind="metering">
<inc type="integer" target="RequestCount"/>
<add type="integer" target="TotalBytes" value="[:$RawData.TotalBytesRead:]"/>
<set type="integer" target="AvgRequestSize" value="[:$ThisData/(number(TotalBytes div RequestCount)):]"/>
</Policy>
<OnError>
<SetLocalData Key="guard" Value=""/>
<ReportError>
<Error>[:[:$LocalData.guard:]$Error:]</Error>
<occurredIn>TQLNotificationCluster</occurredIn>
</ReportError>
<SffReport Name="wsNotificationCluster_OnError"/>
</OnError>
</NewFacetInstance>
</NewPackage> |
Broker Exchange and Bindings
AMQP Vs WS Performance
Using AMQP as a cluster transport does have a cost associated with it due to network involving communication with AMQP borker. Sample Test setup will help us understand the performance cost comparison between the two transports.
Test Setup
| Transport | Instance Type | Boroker Verison | Cluster Size | Cluster Relationship | Database | Cluster Load |
|---|---|---|---|---|---|---|
| AMQP | AWS EC2 m1.medium | RabbitMQ 3.2.4 | 4 | AdvertiseTo | Remote Postgress RDS | Create instances on a model |
Test Results
Cluster Topologies
Choosing the correct data storage strategy depends on the type of application (simple to complex), deployment and interoperability to other platforms. Here are some basic recommended guidelines
...
o For on-the-same-box deployment this is best used with configured port ranges (e.g. sff.server.port=”[8080..8089], see v1.1.1.201704130000). This way your configuration can be completely static and shared across all peers. Any new peer will simply pick up next port from the range and join the cluster. Everything is nice and predictable (i.e. no messing with sff.server.port=0 ephemerals)
...