Assure1 Event Canal microservice

Overview

The Assure1 Event Canal microservice provides the foundation for streaming analytics of events. The Event Canal is responsible for taking real-time events from the Event database and enqueueing them to a Pulsar topic. Multiple microservices can read from this same topic to create analytics pipelines for enrichment, suppresion, north-bound integrations, or machine learning to name a few.

Prerequisites

1. A microservices cluster must be setup. Refer to Microservice Cluster Setup.

2. Apache Pulsar must be installed. Refer to Apache Pulsar microservice.

Setup

su - assure1
export NAMESPACE=a1-zone1-pri
export WEBFQDN=<Primary Presentation Web FQDN> 
a1helm install event-canal assure1/event-canal -n $NAMESPACE --set global.imageRegistry=$WEBFQDN

Default Configuration

Name	Value	Possible Values	Notes
LOG_LEVEL	INFO	FATAL, ERROR, WARN, INFO, DEBUG	Logging level used by application.
STREAM_INPUT	mysql:///Event	Text, 255 characters	If nothing is provided, the application will connect to the Assure1 Shard 1 Event database.
STREAM_OUTPUT	persistent://assure1/event/canal	Text, 255 characters	Apache Pulsar topic path. Topic at end of path may be any text value.

Configurations can be changed by passing the values to the a1helm install prefixed with the configData parent key.

Example of setting the log level to DEBUG

a1helm install ... --set configData.LOG_LEVEL=DEBUG

Examples of setting different input stream options

If nothing is provided as the input stream, the application will connect to the Assure1 Shard 1 Event database.

Note

If using multiple event canal microservices that connect to the same database, a unique client ID per event canal instance is necessary to ensure proper functionality. If this functionality is used, the client ID must be added as an argument to each of the commands below, for example:

a1helm install ... --set configData.STREAM_INPUT="mysql:///?clientID={clientid}"

Example connecting to the Assure1 Event database with a specific shard id:

a1helm install ... --set configData.STREAM_INPUT="mysql:///Event/{shardid}"

Example connecting to the Events table in the Event database on an external database server using a username and password:

a1helm install ... --set configData.STREAM_INPUT="mysql://{username}:{password}@{host}:{port}"

Example connecting to a custom table in a custom database on an external database server using a username and password:

a1helm install ... --set configData.STREAM_INPUT="mysql://{username}:{password}@{host}:{port}/{database}/1/{table}"

Schema

Event data is normalized into a common JSON schema described below. Any custom fields added to the Event.Events table will automatically be included in the JSON, but this example only shows the default fields provided out of the box.

Some fields are converted from their native format into ones appropriate for consumers of the output topic.

{
    "@timestamp": "2006-01-02T15:04:05.999999999Z",
    "_domain":    "fault",
    "_type":      "event",
    "_version":   "5.0.0",
    "event": {
        EventID:        0,
        EventKey:       "",
        EventCategory:  3,
        EventType:      "",
        Ack:            0,
        Action:         "",
        Actor:          "",
        Count:          1,
        Customer:       "",
        Department:     "",
        Details:        {},
        DeviceType:     "",
        Duration:       0.000,
        EscalationFlag: 0,
        ExpireTime:     0,
        FirstReported:  "2006-01-02T15:04:05.999999999Z",
        GeoPath:        {"type": "LineString", "coordinates": [[0, 0], [0, 0]]},
        GeoLocation:    {"type": "Point", "coordinates": [0, 0]},
        IPAddress:      "",
        LastChanged:    "2006-01-02T15:04:05.999999999Z",
        LastReported:   "2006-01-02T15:04:05.999999999Z",
        Location:       "",
        Method:         "",
        Node:           "",
        OrigSeverity:   1,
        OwnerName:      "",
        RootCauseFlag:  0,
        RootCauseID:    0,
        Score:          0,
        Service:        "",
        ServiceImpact:  0,
        Severity:       "Unknown",
        SubDeviceType:  "",
        SubMethod:      "",
        SubNode:        "",
        Summary:        "",
        TicketFlag:     0,
        TicketID:       "",
        ZoneID:         0
    }
}