Row to Column Conversion Involving Uncertain Columns — From SQL to SPL #41
Judith-Data-Processing-Hacks

Judith-Data-Processing-Hacks @judith677

About: esProc SPL is a JVM-based programming language designed for structured data computation, serving as both a data analysis tool and an embedded computing engine. https://www.esproc.com/download-esproc

Joined:
Apr 20, 2022

Row to Column Conversion Involving Uncertain Columns — From SQL to SPL #41

Publish Date: Jun 20
10 3

Problem Description & Analysis:

The query results of the Postgresql database return 6 columns, with the first column being the grouping column, the second column being the Value column within the group, and columns 3–6 being the Key columns within the group. Each record only has one Key column with a value, while the other Key columns are null. The position of the Key with a value for each record is uncertain.

source table
Task: Now we need to keep the grouping column unchanged, convert the details within the group from rows to columns, convert the values of non-null Key columns to new column names, and convert the values of Value column to new column values.

expected results

Solution Highlights:

There are two ways to implement row to column conversion in SQL: one is through functions such as pivot and crosstab, and the other is to use the max group by statement for each new column. However, these two methods require writing new column names, which are the field values of the original table.

When the data changes, SQL also needs to change accordingly, which is very inflexible. If you want a set of code to be applicable to any data, you need to dynamically generate column names using stored procedures or high-level languages, and then execute the previous SQL, which makes the structure much more complex.

The row column conversion function of SPL can be used without writing field values, and a set of code is applicable to any data.

esProc SPL code

A1: Query the database through JDBC.

A2: Use the pivot function to convert this group of records from row to column. The first parameter is the grouping column, the second parameter is the column name of the Key, and the ifn function takes the first non-null member in the set. The third parameter is the Value column.


🚀 esProc SPL FREE Download — Try It Free, Download Now!

Comments 3 total

  • Judith-Data-Processing-Hacks
    Judith-Data-Processing-HacksJun 20, 2025

    Feel free to share your feedback with us!

    Discord
    Reddit

  • Nevo David
    Nevo DavidJun 24, 2025

    Growth like this is always nice to see - makes me wonder, how often do you run into weird columns changing? That’s the stuff that always trips me up.

    • Judith-Data-Processing-Hacks
      Judith-Data-Processing-HacksJun 26, 2025

      Appreciate your thoughts! Dynamic or unpredictable columns come up more often than you’d expect—especially with loosely structured or evolving data sources. That’s exactly where SPL shines by offering flexible ways to handle them. Glad the post resonated with you!

Add comment