Assure1 SMTP Aggregator

Overview

The Assure1 SMTP Aggregator listens for SMTP connections to receive emails, reads the emails sent, parses the results with customizable rules, and creates de-duplicated events within Assure1.

SMTP Aggregator Setup

Default Service

Field	Value
Package Name	coreCollection-app
Service Name	Event SMTP Aggregator
Service Program	bin/core/collection/SMTPd
Service Arguments
Service Description	SMTP Aggregator listens for incoming email messages and aggregates into events
Failover Type	Standalone (Supported: Standalone, Primary/Backup)
Status	Disabled
Privileged	(Checked)

Default Configuration

Name	Value	Possible Values	Notes
BaseRules	collection/event/smtp/base.rules	Text, 255 characters	Relative path to Base Rules.
BranchDir	core/default	Text, 255 characters	relative path to Rules dir
ConnectionSecurity	None (Plaintext)	Plaintext/None / SSL (Secure on Connect) / TLS (Negotiated)	Lowest available secure connection protocol (i.e. if either SSL (Secure on Connect) or TLS (Negotiated) selected, plaintext connection is not available) - NO RELOAD CONFIG SUPPORT
Host	\<SERVER FQDN>	Text, 255 characters	DNS name or IP Address for the SMTP server. Can associate to a particular interface. - NO RELOAD CONFIG SUPPORT
IncludeRules	collection/event/smtp/base.includes	Text, 255 characters	Relative path to Include Rules.
LoadRules	collection/event/smtp/base.load	Text, 255 characters	Relative path to Load Rules.
LogFile	logs/EventSMTPServer.log	Text, 255 characters	Relative path to Log File.
LogLevel	ERROR	OFF, FATAL, ERROR, WARN, INFO, DEBUG	Logging level used by application.
ShardID	1	Integer	Database shard to be used.
Threads	3	Integer	Number of process threads created. The aggregator takes a third of this value (rounded up) for database threads unless overridden by the "DBThreads" application configuration.
Capture	Disabled	Enabled/Disabled	Optional - If enabled, saves the raw message in the Log.
DBThreads		Integer	Optional - Number of database threads to be created. If not specified, defaults to a third (rounded up) of "Threads" application configuration.
DisableDNS	Disabled	Enabled/Disabled	Optional - If enabled, uses IP address instead of hostname for the event Node.
FailoverBufferLimit	0	Integer	Optional - Enables Failover Standby buffer that keeps N-seconds worth of syslogs and replays this buffer when becoming Failover Active. (0=off N=seconds to keep) See Tokens: $buffer and $received
FieldSetFile		Text, 255 characters	Optional - Path to csv file containing custom list of fields that will be used when inserting data. (Requires InsertSQLFile.)
InsertSQLFile		Text, 255 characters	Optional - Path to file containing custom SQL Insert statement for handling of event inserts. (Requires FieldSetFile.)
Port		Integer	Optional - Port for message collection. Defaults to SMTP:25/SMTPS:465 - NO RELOAD CONFIG SUPPORT

Rules

This aggregator uses the Assure1 standard rules architecture, which are 100% Perl syntax. Refer to the following articles to assist in rules creation:

• Core Rules Functions

• Event Rules Functions

• Application Logging

Tokens

The aggregator exposes the following tokens for rules processing.

Token	Description
$AppConfig	Hash reference to the application configuration name-value pairs that were configured. (i.e. use $AppConfig->{'Host'} to retrieve the set value for 'Host'.)
$Event	Reference to the hash that is used to create and insert the Event data into the database. Keys map to the fields within the table used and values assigned are inserted in the database to that field. (e.g. $Event->{'IPAddress'} = '192.0.2.1' to assign the event IP address to '192.0.2.1') At least the 'Node' and 'Summary' fields must be set, or no event is inserted.
$IP	IP of client email was sent from
$DNS	DNS of client email was sent from (if available)
$received	epoch time email was received by the aggregator
$buffer	Flag for if was buffered during standby and was replayed (0 = No, 1 = Yes)
$SubjectOfMessage	Subject of Received Email
$FromEmailAddress	First From Email Address header content (may contain multiple emails)
$ToEmailAddress	First To Email Address header content (may contain multiple emails)
$CCEmailAddress	First CC Email Address header content (may contain multiple emails)
$BodyOfMessage	Body of Received Email. ('text/plain' = PlainBody, 'text/html' = HTMLBody, 'multipart/*' = PlainBody)
$RawEmail	Raw undecoded Email including headers
$ContentType	Content Type of message
$ToAddresses	Array reference to all "To" header contents (usually the same as $Recipients or $ToEmailAddress)
$FromAddresses	Array reference to all "From" header contents (usually the same as $Sender or $FromEmailAddress)
$CCAddresses	Array reference to all "CC" header contents
$ReceivedHeaders	Array reference to all "Received" header contents (contains list of servers mail went through)
$PlainBody	Email body. In text/html content type this contains the plaintext stripped of any HTML tags
$HTMLBody	Email body for text/html content type
$Sender	Sender's email
$Recipients	Recipient email(s) (delimited by , or ;)
$EmailObject	Email::MIME object representing the email. For advanced parsing operations only
$discard_flag	Flag for discard (0 = No, 1 = Yes)
$Count	Message Counter
$CustomHash	Custom key, value cache available across all rules. Contents commonly defined in Load Rules then used in Base or other rules. NOTE: This variable is a shared object and any additional sub hashes or arrays must be shared before use or it will cause the error: "Invalid value for shared scalar". Instantiate the sub hash/array using '&share({})' e.g. $CustomHash->{SubObject} = &share({});
$StorageHash	Internal cache used as the StorageHash option when calling rules functions such as FindDeviceID(). NOTE: The structure of this cache is subject to change! Not recommended for custom global storage or manual manipulation; use $CustomHash.

Administration Details

The following list shows you the technical details you’ll need for advanced administration of the application:

• Package - coreCollection-app

• Synopsis - ./SMTPd [OPTIONS]

• Options:

   -c, --AppConfigID N   Application Config ID (Service, Job, or Request ID)
   -?, -h, --Help        Print usage and exit
  

• Threaded - Multi-Threaded