Aggregate according to Time Interval — From SQL to SPL #27
Judith-Data-Processing-Hacks

Judith-Data-Processing-Hacks @judith677

About: esProc SPL is a JVM-based programming language designed for structured data computation, serving as both a data analysis tool and an embedded computing engine. https://www.esproc.com/download-esproc

Joined:
Apr 20, 2022

Aggregate according to Time Interval — From SQL to SPL #27

Publish Date: May 12
9 3

Problem description & analysis:

A certain database table stores time-series data, with intervals of several seconds between each record.
[Source Table]

Task: Now we need to do a group and aggregation every minute, summarizing data for 5 minutes each time. For example, generate three records in minutes 1, 2, and 3, and summarize the data for minutes 1–5, 2–6, and 3–7 respectively.

expected results

Code comparisons:

SQL

SELECT
    [From], DATEADD(MINUTE, 1, [To]) [To], payload
FROM (
    SELECT
        dt, MIN(dt) OVER(ORDER BY dt ROWS BETWEEN 4 PRECEDING AND CURRENT ROW) [From],
        dt [To], SUM(payload) OVER(ORDER BY dt ROWS BETWEEN 4 PRECEDING AND CURRENT ROW) payload
    FROM (
        SELECT
            DATEADD(MINUTE, DATEDIFF(MINUTE, 0, dt), 0) dt, 
SUM(payload) payload
        FROM #tmstmp
        GROUP BY DATEADD(MINUTE, DATEDIFF(MINUTE, 0, dt), 0)
    ) q
) q
WHERE DATEDIFF(MINUTE, [From], [To]) > 3
Enter fullscreen mode Exit fullscreen mode

SQL needs to implement it using nested subqueries and multiple window functions, which makes the code cumbersome.

SPL: SPL provides the syntax for directly accessing positions.

👉🏻 Try.DEMO

esProc SPL code

A1: Load data.

A2: Group and aggregate by minute.

A3: Generate a new two-dimensional table, where To is taken from the current record, From is taken from 5 minutes after the current record, and the payload summarizes the interval from the current record to the 4th record.

A4: Take from the first item backward to the fifth item.


Free to Try, Powerful to Use — esProc SPL FREE Download.

Comments 3 total

  • Judith-Data-Processing-Hacks
    Judith-Data-Processing-HacksMay 12, 2025

    We'd love to hear your feedback!

    ✍🏻Discord
    ✍🏻Reddit

  • Nevo David
    Nevo DavidMay 13, 2025

    these window functions always get my brain twisted - i swear there's gotta be a simpler way for this kinda rolling stuff you think most people overcomplicate their queries or it's just how sql's built

    • Judith-Data-Processing-Hacks
      Judith-Data-Processing-HacksMay 14, 2025

      Totally feel you on that! Window functions can get pretty tangled, especially for rolling aggregations. I do think SQL’s design makes these patterns a bit verbose by nature. That’s actually why I enjoy exploring SPL—it often lets you express these same calculations in a much simpler way. Might be worth a look if you’re into cleaner, more readable approaches!

Add comment