Quantcast
Channel: User Net Dawg - Stack Overflow
Viewing all articles
Browse latest Browse all 32

Answer by Net Dawg for SQL to Extract Continuous Timestamp Ranges

$
0
0

Another answer to my own question.

Fiddle

WITH cte AS -- the common table expression (CTE)(   -- table projection with all leading and lagging columns to then inform begin and end of failed states     SELECT             LAG(id) OVER (PARTITION BY id ORDER BY timestamp) previous_id,             LAG(status) OVER (PARTITION BY id ORDER BY timestamp) previous_status,             LAG(timestamp) OVER (PARTITION BY id ORDER BY timestamp) previous_timestamp,            id,             status,            timestamp,            LEAD(id) OVER (PARTITION BY id ORDER BY timestamp) next_id,             LEAD(status) OVER (PARTITION BY id ORDER BY timestamp) next_status,             LEAD(timestamp) OVER (PARTITION BY id ORDER BY timestamp) next_timestamp    FROM device_status),-- AND WITH (comment for better readability of multiple queries being introduced on the CTE -- fail start table being defined as FS, fs AS (  -- the order of fail start and, later similarly, failed are preserved in row_number  SELECT ROW_NUMBER() OVER (ORDER BY id, fail_start) r, x.* FROM   (     -- if next _status changed to failed for same id that is the start timestamp.      SELECT id, next_timestamp AS fail_start FROM cte    WHERE next_status = 'Failed' AND next_status IS DISTINCT FROM status AND next_id = id     UNION    -- making an exception for when it is the only fail record    SELECT id, timestamp AS fail_start FROM cte    WHERE previous_status IS NULL AND status = 'Failed'   ) x),-- AND WITH-- fail end table FE, mirror image of FSfe AS (  SELECT ROW_NUMBER() OVER (ORDER BY id, fail_end) r, y.* FROM   (       SELECT id, previous_timestamp AS fail_end FROM cte    WHERE previous_status = 'Failed' AND previous_status IS DISTINCT FROM status AND next_id = id      UNION    SELECT id, timestamp AS fail_end FROM cte    WHERE next_status IS NULL AND status = 'Failed'  ) y)-- simple join of row numbers of FS and FE to generate the required resultSELECT fs.id, fs.fail_start, fe.fail_end FROM fs JOIN fe ON fs.r = fe.r

In this solution, instead of trying to compute a change column, I use both LEAD and LAG functions symmetrically to first line up each record along with its previous and next, when partitioned by ID and ordered by timestamp (an alternative solution of ordering by ID, timestamp proved to be less useful) and then searching for status begin and end in that table concocted specifically to reveal this pattern in a guaranteed manner. Probably best understood through intermediate results of the component subqueries in Fiddle.

Previous-Next (LAG/LEAD View)

Then, I translate in into SQL the logic that any FAILED begin timestamp will be preceded by a non-failed, with exception for the lone failure or first occurrence in the timestamped logs.

Fail Start

The mirror image showing end of Fail status in time will be like so

Fail End

The final answer simply joins the fail start and fail end queries by row number, which is guaranteed to work because of the self projection and, minor, also introduce a sort by ID to completely match result to the original post and accepted answer.

Specifically, the big difference here is that this does not bother generating the proposed change column, sum and group by as originally envisioned in the the post (in order to work backward from the required result).

Hence, I will keep the previous answer as the accepted one.

The results are identical. Performance noted as "too close to call" in a couple of datasets with millions of status records each, with several enumerations of status (not just binary 'Failed' and 'Active' as in the toy dataset).

Further, in favor of the accepted answer, I do still like it because query is much easier to read for me, and possibly others, therefore understand, maintain as code.


Viewing all articles
Browse latest Browse all 32

Latest Images

Trending Articles





Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596344.js" async> </script>