Why S3?
Hook has several native integrations for easy and direct access to your organisation's data. In some cases this may not be possible:
Hook does not have a native integration for the system your organisation uses to store product metrics. Find the list of integrations Hook supports here.
Data must be transformed in a different format compared to how it is originally stored, and therefore should be sent separately to Hook through S3.
A bulk data transfer is required to provide history for training Hook's engagement model. This may occur if there's a change in systems that store important subscriptions or metrics data, e.g. switching between two CRM providers.
Sending data through S3 enables Hook to access valuable data that would otherwise be difficult to access.
Data Format Required by Hook
Data sent to Hook through S3 must meet the following requirements:
File format
CSV or JSONL
Frequency of export
Weekly or daily export i.e. time series with data for each week or each day.
Metric Types (if applicable)
Cumulative metrics (can be added up over time) or stock metrics (snapshot metrics for a specific point in time). Ideally not year-to-date metrics because these reset each year so history for these values is lost.
Examples:
Cumulative metrics: total orders, total downloads, total logins.
Stock metrics: active users (this week), seats occupied (on this day).
Mapping to Account IDs
It must be possible to link each row to an account in your organisation's CRM (e.g. Salesforce account ID or HubSpot company ID). We recommend including a Account ID column that specifies the CRM account ID. Alternatively, each row can be linked to a user ID, tenant ID, opportunity/deal ID etc which could then be linked to an account in your CRM.
Timestamp of data
A timestamp for when the data is relevant for. This should be contained in the filename and as a column in the file.
Example:
The file is named
hook_export_20240301represents data from the 1st March 2024.There is a column in the file called
metric_datewhich contains the value 2024-03-01, to show that this data is from the 1st March 2024.
Column names remain the same in each export
Column names should be the same for every export (regardless of date).
Example:
For metrics dated 1st March 2024, the total number of journeys should have the column name Total Journeys. The column name should NOT be Total Journeys as of Mar-01-2024 (or similar).
Number of columns remain the same in each export
This is required so that Hook can reliably ingest the data. If new columns are added that we do not expect this will cause data import issues.
Example:
The file is named hook_export_20240301 (1st March 2024) contains 10 columns. All of these columns, with the same column names, are included in the following file named hook_export_20240302 (2nd March 2024).
1-2 years of historical data (Customers with engagement models only):
This is needed for subscription and metrics data used to train Hook's engagement model, so that it can learn patterns from past data.
View an example data extract that meets the above requirements here.
How to Send Data to Hook using S3
Format the data following the data requirements described here.
Share a sample extract of this data with your Technical Implementation Consultant. This is an extra check to verify that the data is in the correct format.
Hook will share the following credentials for the S3 bucket securely using LastPass. Remember to keep these credentials secure:
Bucket Name: e.g.
hook-customer-data-production-companyname-imports
Region:eu-west-1Access Key ID:
AKIA...Secret Access Key:
wJalr...Set up automated uploads for these files. Find out more here. Ensure these files are uploaded daily or weekly.
Let the Hook team know once this has been completed.
Speak to your Technical Implementation Consultant for further guidance.
Troubleshooting
Access Denied Error:
Verify you're using the correct credentials
Check you're uploading to the correct bucket name
Confirm you're using the correct profile with the
-profileflag (if using AWS CLI)
Credentials Not Found Error:
Run
aws configure --profileagainCheck credentials are saved in the correct location
File Not Appearing:
Verify upload completed without errors
Check you're looking in the correct bucket/path
Try listing files e.g.
aws s3 ls s3://hook-customer-data/
Example: Data Extract (CSV)
Download the example data extract to see the format required.