logsnarf.app module

Logsnarf application.

Application code to connect the various modules to do something useful..

class logsnarf.app.App(cfg, section_name)

Bases: object

Logsnarf application class.

Currently this is just a convenient way to encapsulate the setup of the various components. It minimizes the use of the configuration object to make for cleaner implementation classes.

Currently a logsnarf.App contains three main components.

logsnarf.snarf.Logsnarf - Monitors directories for log updates, feeds them

linewise to the next in the chain.

logsnarf.uploader.BigQueryUploader - Receieves lines from Logsnarf,

pushes them through the logsnarf.schema.Schema object for munging/verification, adds an insertId to every row, batches them into groups, which get a transaction id (internal use only), and passes them on to the BigQueryService. Partial failures (only some rows) are handled here. Full request failures are generally handled by the BigQueryService. Currently this is where flush-on-exit is handled.

logsnarf.service.BigQueryService - Encapsulates a standard googleapi

BigQueryService along with our project/dataset information and credentials.

Parameters:

• cfg (logsnarf.config.Config) – config object

• section_name (str) – section name from the config that tells us what to do

start()

class logsnarf.app.Options

Bases: Options

optParameters = [['resource_name', 'n', 'logsnarf', 'Resource name'], ['config_file', 'f', None, 'Config file']]

logsnarf.app.install_custom_verifiers(sch, default_domain)

logsnarf.app.install_schema_load_hook(sch)

logsnarf.app.main()