Assure1 Event WebHook Aggregator¶
Overview¶
The Assure1 Event WebHook Aggregator is a generic daemon message listener that receives messages from HTTP(S) requests, parses the results with customizable rules, and creates de-duplicated events within Assure1.
WebHook Aggregator Setup¶
-
Review the logic in the rules files referenced in the configuration to see the processing that will be done when HTTP requests are received:
-
"LoadRules" will be executed during application startup to load data that might be needed during processing.
-
"IncludeRules" will be read during application startup to load additional files that might be called during processing.
-
"BaseRules" will be executed for each device that is selected based on the configuration.
Update the logic as needed.
-
-
Enable the default Service, unless a specific configuration option is needed.
-
Configure devices to send HTTP(S) messages to the server where the WebHook Aggregator has been started.
Default Service¶
| Field | Value |
|---|---|
| Package Name | coreCollection-app |
| Service Name | Event WebHook Aggregator |
| Service Program | bin/core/collection/WebHookd |
| Service Arguments | |
| Service Description | Gathers events over HTTP |
| Failover Type | Standalone (Supported: Standalone, Primary/Backup) |
| Status | Disabled |
| Privileged | (Checked) |
Default Configuration¶
| Name | Value | Possible Values | Notes |
|---|---|---|---|
| BaseRules | collection/event/webhook/base.rules | Text, 255 characters | Relative path to Base Rules. |
| BranchDir | core/default | Text, 255 characters | relative path to Rules dir |
| Host | * | Text, 255 characters | DNS name or IP Address of the webhook aggregator server. Can associate to a particular interface. - NO RELOAD CONFIG SUPPORT |
| IncludeRules | collection/event/webhook/base.includes | Text, 255 characters | Relative path to Include Rules. |
| LoadRules | collection/event/webhook/base.load | Text, 255 characters | Relative path to Load Rules. |
| LogFile | logs/EventWebHook.log | Text, 255 characters | Relative path to Log File. |
| LogLevel | ERROR | OFF, FATAL, ERROR, WARN, INFO, DEBUG | Logging level used by application. |
| Port | 10080 | Integer | Port for webhook server to collect messages. - NO RELOAD CONFIG SUPPORT |
| ShardID | 1 | Integer | Database shard to be used. |
| Threads | 3 | Integer | Number of process threads created. The aggregator takes a third of this value (rounded up) for database threads unless overridden by the "DBThreads" application configuration. |
| Capture | Disabled | Enabled/Disabled | Optional - If enabled, the application will log all of the fields of the processed message as it sent into the database. |
| ConnectionSecurity | None (Plaintext) | None (Plaintext) / SSL (Secure on Connect) / TLS (Negotiated) | Optional - Connection protocol required for incomming connections (i.e. if either SSL (Secure on Connect) or TLS (Negotiated) selected, non-secure connections are not available). If not specified, defaults to "None (Plaintext)". - NO RELOAD CONFIG SUPPORT |
| DBThreads | Integer | Optional - Number of database threads to be created. If not specified, defaults to a third (rounded up) of "Threads" application configuration. | |
| DisableDNS | Disabled | Enabled/Disabled | Optional - If enabled, uses IP address instead of hostname for the event Node. |
| FailoverBufferLimit | 0 | Integer | Optional - Enables Failover Standby buffer that keeps N-seconds worth of syslogs and replays this buffer when becoming Failover Active. (0=off N=seconds to keep) See Tokens: $buffer and $received |
| FieldSetFile | Text, 255 characters | Optional - Path to csv file containing custom list of fields that will be used when inserting data. (Requires InsertSQLFile.) | |
| InsertSQLFile | Text, 255 characters | Optional - Path to file containing custom SQL Insert statement for handling of event inserts. (Requires FieldSetFile.) |
Best Practices¶
The following list shows the best practices for working with this application:
-
Depending on the server specifications and loads, the number of the threads that the aggregator uses may need to be increased. Assure1 believes a good starting point for threads is 3 * (# of CPU Cores).
-
By default, HTTP request messages are sent in a carriage return delimited format.
Rules¶
This aggregator uses the Assure1 standard rules architecture, which are 100% Perl syntax. Refer to the following articles to assist in rules creation:
Tokens¶
The aggregator exposes the following tokens for rules processing.
| Token | Description |
|---|---|
| $AppConfig | Hash reference to the application configuration name-value pairs that were configured. (i.e. use $AppConfig->{'Host'} to retrieve the set value for 'Host'.) |
| $Event | Reference to the hash that is used to create and insert the Event data into the database. Keys map to the fields within the table used and values assigned are inserted in the database to that field. (e.g. $Event->{'IPAddress'} = '192.0.2.1' to assign the event IP address to '192.0.2.1') At least the 'Node' and 'Summary' fields must be set, or no event is inserted. |
| $received | Epoch time request was received by the aggregator |
| $buffer | Flag for if was buffered during standby and was replayed (0 = No, 1 = Yes) |
| $node | DNS Resolved Source of HTTP request |
| $ip | IP of Source of HTTP request |
| $content | The (decoded if needed) content of the HTTP request |
| $rawContent | The raw content of the HTTP request |
| $headers | The headers sent with the HTTP request |
| $uri | The path that the HTTP request is sent to. |
| $discard_flag | Flag for discard (0 = No, 1 = Yes) |
| $count | Message Counter |
| $CustomHash | Custom key, value cache available across all rules. Contents commonly defined in Load Rules then used in Base or other rules. NOTE: This variable is a shared object and any additional sub hashes or arrays must be shared before use or it will cause the error: "Invalid value for shared scalar". Instantiate the sub hash/array using '&share({})' e.g. $CustomHash->{SubObject} = &share({}); |
| $StorageHash | Internal cache used as the StorageHash option when calling rules functions such as FindDeviceID(). NOTE: The structure of this cache is subject to change! Not recommended for custom global storage or manual manipulation; use $CustomHash. |
Administration Details¶
The following list shows the technical details need for advanced administration of the application:
-
Package - coreCollection-app
-
Synopsis -
./WebHookd [OPTIONS] -
Options:
-c, --AppConfigID N Application Config ID (Service, Job, or Request ID) -?, -h, --Help Print usage and exit -
Threaded - Multi-Threaded