Distributed Analytics Service (DAS)

1 Distributed Analytics Service Overview
2 DAS Functional Components
3 DAS Features
4 Getting Started with DAS
5 Future Features

Distributed Analytics Service Overview

A-Stack Distributed Analytics Service (DAS) allows TQL developers to seamlessly integrate a number of python-based regression and classification algorithms from into their application.

DAS Functional Components

Component Name	Description	Examples

Component Name

Description

Examples

Pre-defined set of A-Stack Activities

A set of simple activities which collects commands from DASScript and executes in a python runtime as part of Exec Activity

DAS Configuraton Model

Configuration parameters for DAS

#
scope(fid: dasact):
 $vopak_dasconfig =
  DASConfig:
    DASContextID: "SteamHist-1"
    ModelOutputHandler:
      facetID: "steamws"
      ActionName: "HandleModelOutput_Action"

    InputDataSource:
      TQLFacetID: steamws
      ColumnNames: "steamConsumedValue,deltaTemp,Valve_open_Temp"
      IndexNames: "deltaTemp"
      FindQuery:
       Find(as: "$none:$none:$none", Only: "steamConsumedValue, deltaTemp, Valve_open_Temp"):
         SteamConsHistory:
           SteamHistId(ne: "")
    DASScript:
      @: 'resources/SteamPredict/spaces/DASPolyline.das.cdm'

DAS Python SDK

Offers basic functions that can called from DASConfig’s DAS Script.

Functions:

broadcastmsg
savemodel
readmodel

Currently Supports following libraries

import numpy as np
from numpy import nan
import datetime
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import MinMaxScaler
from sklearn.pipeline import make_pipeline
import statsmodels.api as sm
from sklearn.metrics import r2_score
from sklearn.metrics import mean_squared_error
from sklearn.metrics import mean_absolute_error
import statsmodels.api as sm
from sklearn.model_selection import train_test_split
from collections import namedtuple
from sklearn import linear_model

DAS Features

DAS Features description

Feature Category	Feature Description	Example

Feature Category	Feature Description	Example
Feeding Training or Testing (To-be-Predicted) Data	InputDataSource - DAS Provides InputDataSource parameter as part of the DASConfig Parameter	`InputDataSource: TQLFacetID: steamws ColumnNames: "steamConsumedValue,deltaTemp,Valve_open_Temp" IndexNames: "deltaTemp" FindQuery: Find(as: "$none:$none:$none", Only: "steamConsumedValue, deltaTemp, Valve_open_Temp"): SteamConsCurrent: SteamId(ne: "")`
Training Data Preparation	Data_Shape - DAS Provides Data_Shape Section as part of the DASConfig’s DASScript where TQL developers can add their Data Shaping code. Data_Split_TrainAndValidate - DAS Provides Data_Split_TrainAndValidate Section as part of the DASConfig’s DASScript where TQL developers can add further steps their Data Shaping code.
Providing Algorithm Details	PredictiveModels - DAS Provides PredictiveModels Section as part of the DASConfig’s DASScript where TQL developers can add specific algorithm steps
Handle Model Output
Saving Trained Model for Prediction re-use
Reading Saved model
Real-time viewing of algorithm execution logs	All broadcastmsg statements from DAS Script are published to DASLog websocket endpoint

Getting Started with DAS

Future Features

Seamless stream ingestion into DAS
Seamless stream egres to Python from DAS
Scalable Data Preprocessing
Analytics Model versioning
Analytics library exposing functional interface to BML (python sdk)
Host and run external models