ServicesPlatformData Streamer

Stream types

Through stream types, you can configure the type of content delivered by the data stream. Depending on your subscription, you may choose between receiving event data, usage data, or Flow Logs (flow data).

Streaming event and usage data within one stream is possible but not advised, as these data records don’t share a common schema definition.

Event data

The emnify system generates several types of events. These events allow you to track notable system occurrences based on behavior.

Some common use cases for events on emnify include:

  • Triggers for custom business processes (for example, authentication or custom usage limitations configured on the emnify Portal)
  • Monitoring (for example, SIM or data connection lifecycles)
  • Input for custom billing systems (that is, updating billing configuration, processing invoices, etc.)

Learn more about the available event types and how to use them in the Event documentation.

Usage data

Usage data records get generated when devices consume data or SMS services. These records provide information about the SIM, used service, visited network, volumes, and cost.

Usage data streams are often used for monitoring and analyzing data consumption and as input for custom billing systems.

Each usage data record contains information about the:

  • Device (organization, SIM, IMSI)
  • Time (start and end time)
  • Price applied for rating (data plan, coverage policy, coverage area)
  • Mobile network operator used for the service (operator, country)
  • Type of service (SMS or data)
  • Consumed service volumes (down- and upstream, total volume)
  • Costs (amount, currency)

Traffic types

IDDescription
5Data
6SMS

Data

Data usage records are created:

  • Every 45 seconds for open PDP contexts (when at least 100 KB of data is consumed)
  • After the PDP context is closed
1[
2    {
3        "cost": 0.00188512,
4        "id": 333333333333333,
5        "operator": { // network
6            "id": 5,
7            "name": "Telefonica O2",
8            "mnc": "07",
9            "country": {
10                "id": 74,
11                "mcc": "262",
12                "name": "Germany"
13            }
14        },
15        "organisation": {
16            "id": 12345,
17            "name": "Example Organization"
18        },
19        "tariff": { // data plan
20            "id": 1,
21            "name": "Internal Test Data Plan",
22            "ratezone": { // coverage area
23                "id": 2,
24                "name": "Area 2"
25            }
26        },
27        "traffic_type": { // usage type
28            "id": 5,
29            "description": "Data"
30        },
31        "endpoint": { // device
32            "id": 123456789,
33            "name": "Example Device",
34            "ip_address": "192.0.2.7",
35            "tags": "V1",
36            "imei": "8677300511111142",
37            "balance": null
38        },
39        "imsi": "295050901064821",
40        "volume": {
41            "total": 0.219933,
42            "rx": 0.172848,
43            "tx": 0.047085
44        },
45        "start_timestamp": "2022-04-26T11:53:43Z",
46        "sim": {
47            "id": 3324192,
48            "iccid": "89883030000080139311",
49            "msisdn": "423663920123456",
50            "production_date": "2020-09-09T06:42:59Z"
51        },
52        "currency": {
53            "id": 1,
54            "code": "EUR",
55            "symbol": "€"
56        },
57        "end_timestamp": "2024-02-15T14:56:15.260Z",
58        "imsi_id": 9624042,
59        "session_id": "722aeb56-c4de-4753-ac23-4f99c1980c5e"
60    }
61]
API reference

Data Streamer usage object

SMS

Usage records for SMS are created when an SMS is successfully delivered either:

  • From the device ("rx": 1)
  • Towards the device ("tx": 1)
1[
2    {
3        "cost": 0.07,
4        "id": 333333333333333,
5        "operator": { // network
6            "id": 5,
7            "name": "Telefonica O2",
8            "mnc": "07",
9            "country": {
10                "id": 74,
11                "mcc": "262",
12                "name": "Germany"
13            }
14        },
15        "organisation": {
16            "id": 12345,
17            "name": "Example Organization"
18        },
19        "tariff": { // data plan
20            "id": 1,
21            "name": "Internal Test Data Plan",
22            "ratezone": { // coverage area
23                "id": 1,
24                "name": "Area 1"
25            }
26        },
27        "traffic_type": { // usage type
28            "id": 6,
29            "description": "SMS"
30        },
31        "endpoint": { // device
32            "id": 123456789,
33            "name": "Example Device",
34            "ip_address": "192.0.2.88",
35            "tags": "V1",
36            "imei": "8677300511111142",
37            "balance": null
38        },
39        "imsi": "901430111111111",
40        "volume": {
41            "total": 1,
42            "rx": 1,
43            "tx": 0
44        },
45        "start_timestamp": "2022-04-26T13:13:56Z",
46        "sim": {
47            "id": 123456,
48            "iccid": "89883030000080139312",
49            "msisdn": "423663920123456",
50            "production_date": "2020-09-09T06:42:59Z"
51        },
52        "currency": {
53            "id": 1,
54            "code": "EUR",
55            "symbol": "€"
56        },
57        "end_timestamp": "2024-02-15T14:56:15.260Z",
58        "imsi_id": 9624042,
59        "session_id": "beafcbbf-48b7-49c5-b493-fce36bad466a"
60    }
61]
API reference

Data Streamer usage object

Flow Logs (flow data) Beta

Flow Logs (also referred to as flow data) capture metadata about IP traffic generated by devices across emnify’s mobile core. Use them for observability, security, troubleshooting, and compliance use cases.

Flow Logs streaming is in beta. Flow Logs are available for all Data Streamer integrations except Keen and Datadog, where they’re disabled.

The Flow Logs stream contains enriched flow records, including:

  • Flow metadata - 5-tuple, protocol, direction, timestamps, bytes up/down, and NAT context where applicable
  • Domain attribution - DNS answers and server name observations such as TLS SNI or HTTP Host, where enrichment is possible
  • Device and network context - organization, endpoint, APN, MCC/MNC, country, and session metadata where available

Flow Log records are emitted at stages of a flow’s lifecycle, indicated by the event_type field:

  • flow_start - first record emitted when a new flow is observed
  • flow_update - periodic snapshot of an active flow
  • flow_end - final record emitted when the flow terminates, including release details

Learn more about the available flow data event types in the Network Flow Logs documentation.

Configuration

Flow Logs currently provide a single canonical stream without selectable event filtering or stream options.

The stream includes a buffering window to bridge short interruptions, such as temporary outages or pauses.