logsnarf.service module
BigQuery service.
- class logsnarf.service.BigQueryService(project_id, dataset, creds, reactor=None, debug=False)[source]
Bases:
objectA fairly basic wrapper around the google BigQuery API.
- Parameters:
project_id (int) – Numeric project id
dataset (str) – Dataset name
creds (oauth2client.Credentials) – oauth2 credentials
reactor (twisted.internet.reactor) – twisted reactor object
- createTable(name, table_schema)[source]
Create a BigQuery table
- Parameters:
- Raises:
googleapiclient.errors.HttpError – if a HTTP error occurs
ssl.SSLError – if an SSL based error occurs
- property http
Creates and authorizes a httplib2.Http() instance
- insertAll(table, table_schema, data, upload_id=None)[source]
Insert rows into a table. Create the table if necessary.
This is typically called in the main thread.
- Parameters:
table (str) – table to insert data to.
table_schema (list) – list of fields, representing the table schema this can be ommitted if the table already exists.
data (list(dict)) – the rows to be intersted. usually a list of dicts.
upload_id (str) – unique identifier for this upload, only used internally for logging. one is generated if not supplied
- Returns:
a deferred for the results, the form of which is described at https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll#response-body
- Return type:
- insertAll_s(table, table_schema, data, upload_id=None)[source]
Synchronous version of
insertAll().- Parameters:
table (str) – table to insert data to.
table_schema (list) – list of fields, representing the table schema this can be ommitted if the table already exists.
data (list(dict)) – the rows to be intersted. usually a list of dicts.
upload_id (str) – unique identifier for this upload, only used internally for logging. one is generated if not supplied
- Returns:
Results of the insert.
- Return type:
https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll#response-body
- property service
Creates a BigQuery service.