Collect array of distinct values using collect_set and concatenate it with delimiter ‘,’. This will produce list without outer quotes 2009-02-01′,’2009-04-01 and in the second script add outer quotes ‘ also, or you can add them in the first query. And when executing in inline sql (-e option) you do not need to pass hiveconf … Read more
Specify split column manually. Split column is not necessarily equal to PK. You can have complex PK and some int Split column. You can specify any integer column or even simple function (some simple function like substring or cast, not aggregation or analytic). Split column preferably should be evenly distributed integer. For example if your … Read more
I find it easy to understand certain concepts with help of diagrams and this is certainly one of them. Lets assume that our secondary sorting is on a composite key made out of Last Name and First Name. With the composite key out of the way, now lets look at the secondary sorting mechanism The … Read more