Manual index and conso on Data Hub 5

Manual index and Manual conso feature allows to enter either indexes or manual consumption through the UI. Based on these raw data, the system generates, through a calculated variable named 'General Consumption', a time series which represents the average daily consumption.

Manual index behavior

A source of type 'Manual index' allows to enter indexes. Maximum 1 index per day is allowed. As soon as at least 2 indexes are entered in the system, a 'general consumption' is calculated. This time series represents the average daily consumption.

Example :

The two following indexes are entered, through the UI, on the manual index raw variable :

The following datapoints are automatically generated on the 'General consumption' calculated variable :

• Date : 2022-04-04, ConsumptionValue : 500
• Date : 2022-04-05, ConsumptionValue : 500

Manual conso behavior

A source of type 'Manual conso' allows to enter consumption values. Maximum 1 consumption per day is allowed. As soon as at least 2 consumptions are entered in the system, a 'general consumption' is calculated. This time series represents the average daily consumption.

Example :

The two following conso are entered, through the UI, on the manual conso raw variable :

The following datapoints are automatically generated on the 'General consumption' calculated variable :

• Date : 2022-04-04, Value : 7.5
• Date : 2022-04-05, Value : 7.5

Create a Manual index or conso Source

To create a manual index or manual conso source:

• go on the source page
• click on the 'Add source' button
• choose the site and 'custom source'
• In the General information tab, enter the required parameters and choose Manual index or Manual Conso as 'Source Type'
• Click on Save

The source is created along with the variables.

A tab Manual Index or Manual Conso is available and allows to manage data on the raw variable.

Raw Data management

Display data

The raw data (index or manual conso) are displayed in the source tab 'Manual index' or Manual conso'.

All the raw data ingested and contained in the range from 'Now - 5 years' to 'Now + 2 years' are displayed in this tab.

Existing data out of this range are stored in dataHub but are not displayed and cannot be managed through the tab.

Example : If Now = 01-01-2022 => all raw data with a date between 01-01-2017 and 01-01-2024 will be displayed in the tab

Maximum one datapoint per day is expected and at a time equals to 00:00:00 (source time zone). If datapoints are ingested at a different time, they are displayed but in an ‘invalid state’.

Please, note that the data are displayed in the current source's timezone.

Insertion

The UI allows to insert maximum one datapoint per day on the raw variable (manual index or manual conso) and ingests it automatically at 00:00:00.000 (source's time zone).

A data can be inserted in the following range :

• Minimum Date: Now-5years
• Maximum Date : Now + 2 years

To add a new index or conso :

• click on the Add Index button
• choose a date
• enter a value
• click on Add
• click on Save

The data is first displayed in a 'processing' state and cannot be modified. As soon as the datapoint has been effectively ingested, it can be edited or deleted.

Please, note that the data are inserted in the current source's timezone.

Tip

Index and Conso can also be uploaded in bulk through the OpisenseManualLoad File. More info here

Update

The date and/or the value can be updated through the UI for each index/conso.

To edit an index/conso :

• go to the Manual Index or Manual Conso tab
• click on the edit icon related to the data to update
• Change either the date and/or the value
• Click on Update
• Click on Save to apply the modifications

After saving the modification, the index/conso is in a processing state until the new date/value are correctly stored in the system.

The related General consumption calculated variable is automatically triggered by the update of the raw variable and the data are recomputed.

Deletion

Index/Conso data can be deleted one by one or in bulk.

To delete an index/conso data :

• go on the Manual Index or Manual Conso tab
• click on the trash icon related to the data to delete
• Click on Save

The deletion is performed immediately. There is no processing state in this case.

To delete several indexes/conso, several checkboxes can be selected. Click then on the delete icon which has appeared:

To delete all the indexes/conso, use the 'select all' checkbox :

Only the index/conso in the display range (From Now - 5 years To Now + 2 years) can be deleted through the UI. Data out of this range can be deleted through the API (See Delete Data).

Warning

Please, be aware that the deletion of index/conso is irreversible. Once deleted, the data cannot be retrieved anymore.

Invalid Datapoints

Datapoints can be in an invalid state, meaning they cannot be managed through the UI anymore.

This can happen in two cases (to avoid) :

Case 1 : the source's timezone has been updated after the ingestion of data

If the time zone of the source has been updated, all the index/conso ingested before this modification are tagged as invalid Datapoint.

Case 2 : the time of the datapoint is not equals to 00:00:00

If index/conso have been ingested through another channel that the UI (standard file for example) and the time of the datapoints are not equal to 00:00:00 (1 datapoint per day max), the datapoints concerned are tagged as invalid Datapoint

In both cases, the datapoints cannot be updated or deleted through the UI anymore. The behavior of the related General consumption calculated variable is then unpredictable.

We strongly recommend to :

• Not update the time zone of the source
• ingest maximum one datapoint per day, at 00:00:00

Required Configuration

The raw and calculated variables are automatically created after the source creation (through UI) with all the configuration needed for the feature to work correctly.

However, it is possible to create separately, through the API, the source and the two variables needed for the Manual Index / Conso feature.

Source configuration

By using the request POST/source, a manual index or conso source can be created. The following properties are mandatory :

{
    "siteId": 1111,
    "siteName": "NameOfTheSite",
    "energyTypeId": 1111,
    "energyUsageId": 1111,
    "name": "NameOfTheSource",
    "sourceTypeId": 25,
    "timeZoneId": "TimeZoneOfTheSource"
  }

In case of a Manual Conso source, the sourceTypeId must be equals to 25. For a Manual Index source, the sourceTypeId must be equals to 4

Variables Configuration

Raw Variable

In case of a Manual Conso,

• the mappingConfig of the Manual Conso variable must be equal to "Manual Conso"
• the quantityType to 2

For a Manual Index,

• the mappingConfig of the Consumption index variable must be equal to "Consumption index"
• the quantityType to 3

{
"name": "Consumption index",
"variableTypeId": 130,
"granularity": 1,
"granularityTimeBase": 4,
"quantityType": 3,
"mappingConfig": "Consumption index",
"aggregate": 3
}

Calculated Variable

The calculated variable named ‘General consumption’ must have the following properties :

{
    "calculated": {
      "friendlyName": "General consumption",
      "calculatedVariableFormulas": [
        {
          "formula": "opinum_consumption_from_manual_consumption(inputVariables$index,targetVariable)",
          "variables": [
            {
              "alias": "index",
              "siteId": 1111,
              "sourceId": 1111,
              "variableId": 1111,
              "granularity": 0,
              "periodType": 3,
              "period": 2,
              "periodTimeBase": 99999,
              "unitId": 8,
              "canBeUsedAsATrigger": true
            }
          ],
          "entities": [],
          "constants": []
        }
      ],
      "isCalculatedAtRuntime": false,
      "impactingVariableIds": [
        11111
      ],
      "engineId": "R:V4.1.1"
    },
    "name": "General consumption",
    "sourceId": 11111,
    "variableTypeId": 0,
    "isDefault": false,
    "unitId": 8,
    "granularity": 1,
    "granularityTimeBase": 4,
    "quantityType": 2,
    "mappingConfig": "General consumption of 1111",
    "aggregate": 1
  }

Where :

• siteId = related siteId
• sourceId = related sourceId
• variableId = Id of the related raw variable
• mappingConfig = "General consumption of Id of the raw variable"
• formula :
  • Manual Conso = "opinum_consumption_from_manual_consumption(inputVariables$index,targetVariable)"
  • Manual Index = "opinum_consumption_from_index(inputVariables$index,targetVariable)"

If the source and/or variables settings are invalid, the manual index/conso tab is disabled with a warning message.

Users Configuration

Source and variables creation

Only managers and users with 'Admin' permissions on 'Sources' are able to create Manual Index/Conso sources through the UI.

Data Management

To be able to access the data of a Manual Index/Conso source, the user needs access to the site and the source itself.

For more information on the users configuration in Datahub, See Users Rights

To insert an index/conso, the role DataPusher on the OpisenseStandard device is required.

To delete an index/conso, the permissions 'Admin' on the 'Sources' is required

To edit an index/conso, both DataPusher on OpisenseStandard AND the 'Admin' permissions on 'Sources' are needed.