Temporal joins The memory consumed (state size) of a temporal join is proportional to the number of events in the temporal wiggle room of the join, which is event input rate multiply by the wiggle room size. SELECT count(.) FROM input GROUP BY clusterid, tumblingwindow (minutes, 5) In order to ameliorate issues caused by high cardinality in the previous query, you can send events to Event Hub partitioned by clusterid, and scale out the query by allowing the system to process each input partition separately using PARTITION BY as shown in the example below: SELECT count(.) FROM input PARTITION BY PartitionId GROUP BY PartitionId, clusterid, tumblingwindow (minutes, 5) Once the query is partitioned out, it is spread out over multiple nodes.

For example, in the following query, the number associated with clusterid is the cardinality of the query. Instead, the memory consumed is proportional to the cardinality of the data, or the number of groups in each time window. Temporal analytic functions: ISFIRST, LAST, and LAG with LIMIT DURATION The following factors influence the memory used (part of streaming units metric) by Stream Analytics jobs: Windowed aggregates The memory consumed (state size) for a windowed aggregate is not always directly proportional to the window size. Temporal joins: JOIN with DATEDIFF function. Windowed aggregates: GROUP BY of Tumbling, Hopping, and Sliding windows.

Streaming Due Fantagenitori