Spark Map Data type Issue

I'm having the below issue when selecting from a view using spark. And the view definition has one or more tables with Map data type. How do we handle this?

SQL:

select XXX_ID,count(1) as mtrc_value 
FROM GLOBAL.DDM_VW_CPT
WHERE XXX_ID > date_sub(current_date,8) group by XXX_ID

Error : Exception Occurred -

Cannot have map type columns in DataFrame which calls set operations(intersect, except, etc.), but the type of column event_values is map;;

1 answer

  • answered 2018-07-11 06:06 Kishore

    XXX_ID should be in group by:

    select XXX_ID,count(1) as  mtrc_value 
    FROM GLOBAL.DDM_VW_CPT 
    WHERE XXX_ID > date_sub(current_date,8) group by XXX_ID