Webhook Standard data file

To receive a feedback once a Standard data file upload is proceed on your account, just set up a webhook in Data Hub.

Available information

This webhook will allow you to receive the following information each time a Standard data file is uploaded on your account:

  • State of the file processing: Ok, no new data to process, errors in the processing
  • File name: name of the file uploaded
  • File ID: ID of the file uploaded in the storage
  • Total number of ingested datapoints
  • Duplicate data in the files:
    • Number of duplicate data rows
    • Duplicate data rows ID
  • Non ingested datapoints:
    • Total data row not ingested
    • Not ingested data rows
    • Reason of non ingestion

Common errors

The system will throw a process error (without details) in the following cases (non-exhaustive list):

  • Empty file
  • Wrong separator
  • Single quotes
  • Missing mandatory columns ('date', 'value' or columns to identify the variable)
  • Missing mandatory value in the first row in the file (the one just after the headers line)
  • Wrong value type in source id (e.g.: string)

Please, note that to identify the variable, the system chooses a combination of information based on what is provided in the first row (after the headers row). Once the identification way is set, the same columns are used for all the rows. E.G.: if in the first row, there is a variable id => the identification will be used this info and search for it in other rows. If in the second row, there is no var id => data not loaded for this row (even if mapping config and source id available).

The system will throw an error related to a row, in the following cases (non-exhaustive list):

  • VariableNotFound:
    • Incorrect or missing info needed to identify the variable. Examples:
    • Non existing var id o the account
    • No match between source info and mapping config
    • Empty mapping config
  • InvalidDate:
    • Non existing date (e.g. 30-02-2020)
    • Wrong format
    • Wrong type value (ex: string)
  • InvalidValue:
    • No Value
    • Wrong type value in value column (data which is not a double)
  • SourceNotFound:
    • Source ID is an integer but does not match with an existing source ID on the account

Some cases do not generate an error, the rows are simply ignored:

  • Var id column empty (if the identification way uses this column)
  • Source id column empty (if the identification way uses this column)
  • Wrong type value in var id (e.g.: string)
  • No date
Tip

Learn how to manage webhooks and discover all kind of webhooks available in Data Hub.