How to Order/Sort the structType based on the field names in Java Spark 2.1.1Spark - How to add a StructField at the beginning of a StructType in scalaDataFrame partitionBy on nested columnsSpark Struct structfield names getting changed in UDFHow to group by on a field inside an array of an array of records?How do I apply schema with nullable = false to json readingSpark read avroSpark custom sort column in JavaHow to cast all columns of a DataFrame (with Nested StructTypes) to string in SparkSpark FlatMapGroupsWithStateFunction throws cannot resolve 'named_struct()' due to data type mismatch 'SerializeFromObjectPySpark sql dataframe pandas UDF - java.lang.IllegalArgumentException: requirement failed: Decimal precision 8 exceeds max precision 7
Why do ¬, ∀ and ∃ have the same precedence?
Is this toilet slogan correct usage of the English language?
What are some good ways to treat frozen vegetables such that they behave like fresh vegetables when stir frying them?
How to make money from a browser who sees 5 seconds into the future of any web page?
How to draw a matrix with arrows in limited space
Can I say "fingers" when referring to toes?
Creating two special characters
Is it necessary to use pronouns with the verb "essere"?
Does the reader need to like the PoV character?
What is going on with gets(stdin) on the site coderbyte?
What does "Scientists rise up against statistical significance" mean? (Comment in Nature)
Microchip documentation does not label CAN buss pins on micro controller pinout diagram
Non-trope happy ending?
What kind of floor tile is this?
Does "he squandered his car on drink" sound natural?
Are Captain Marvel's powers affected by Thanos breaking the Tesseract and claiming the stone?
Is there a nicer/politer/more positive alternative for "negates"?
Why should universal income be universal?
When were female captains banned from Starfleet?
Does Doodling or Improvising on the Piano Have Any Benefits?
C++ check if statement can be evaluated constexpr
Is there any evidence that Cleopatra and Caesarion considered fleeing to India to escape the Romans?
US tourist/student visa
How do I tell my boss that I'm quitting soon, especially given that a colleague just left this week
How to Order/Sort the structType based on the field names in Java Spark 2.1.1
Spark - How to add a StructField at the beginning of a StructType in scalaDataFrame partitionBy on nested columnsSpark Struct structfield names getting changed in UDFHow to group by on a field inside an array of an array of records?How do I apply schema with nullable = false to json readingSpark read avroSpark custom sort column in JavaHow to cast all columns of a DataFrame (with Nested StructTypes) to string in SparkSpark FlatMapGroupsWithStateFunction throws cannot resolve 'named_struct()' due to data type mismatch 'SerializeFromObjectPySpark sql dataframe pandas UDF - java.lang.IllegalArgumentException: requirement failed: Decimal precision 8 exceeds max precision 7
I have a structType schema, i need to order / sort the schema based on the field name, below i am sharing the StructType.
StructType schema = StructType(StructField(zzz,StringType,true),
StructField(kkk,StringType,true),
StructField(aaa,StringType,true),
StructField(lll,StringType,true))
i would like to get the above structType, as below
StructType schema = StructType(StructField(aaa,StringType,true),
StructField(kkk,StringType,true),
StructField(lll,StringType,true),
StructField(zzz,StringType,true))
apache-spark apache-spark-sql apache-spark-dataset
asked Mar 7 at 4:50
svk 041994svk 041994
587
add a comment |
I have a structType schema, i need to order / sort the schema based on the field name, below i am sharing the StructType.
StructType schema = StructType(StructField(zzz,StringType,true),
StructField(kkk,StringType,true),
StructField(aaa,StringType,true),
StructField(lll,StringType,true))
i would like to get the above structType, as below
StructType schema = StructType(StructField(aaa,StringType,true),
StructField(kkk,StringType,true),
StructField(lll,StringType,true),
StructField(zzz,StringType,true))
apache-spark apache-spark-sql apache-spark-dataset
asked Mar 7 at 4:50
svk 041994svk 041994
587
add a comment |
I have a structType schema, i need to order / sort the schema based on the field name, below i am sharing the StructType.
StructType schema = StructType(StructField(zzz,StringType,true),
StructField(kkk,StringType,true),
StructField(aaa,StringType,true),
StructField(lll,StringType,true))
i would like to get the above structType, as below
StructType schema = StructType(StructField(aaa,StringType,true),
StructField(kkk,StringType,true),
StructField(lll,StringType,true),
StructField(zzz,StringType,true))
apache-spark apache-spark-sql apache-spark-dataset
asked Mar 7 at 4:50
svk 041994svk 041994
587
I have a structType schema, i need to order / sort the schema based on the field name, below i am sharing the StructType.
StructType schema = StructType(StructField(zzz,StringType,true),
StructField(kkk,StringType,true),
StructField(aaa,StringType,true),
StructField(lll,StringType,true))
i would like to get the above structType, as below
StructType schema = StructType(StructField(aaa,StringType,true),
StructField(kkk,StringType,true),
StructField(lll,StringType,true),
StructField(zzz,StringType,true))
apache-spark apache-spark-sql apache-spark-dataset
apache-spark apache-spark-sql apache-spark-dataset
asked Mar 7 at 4:50
svk 041994svk 041994
587
asked Mar 7 at 4:50
svk 041994svk 041994
587
asked Mar 7 at 4:50
svk 041994svk 041994
587
asked Mar 7 at 4:50
svk 041994svk 041994
587
asked Mar 7 at 4:50
svk 041994svk 041994
587
587
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In java,
new StructType(Stream.of(schema.fields())
.sorted(Comparator.comparing(StructField::name))
.collect(Collectors.toList()).toArray(new StructField[schema.fields().length]))
In scala
You can do something like,
var schema = StructType(Seq(StructField("zzz",StringType,true),
StructField("kkk",StringType,true),
StructField("aaa",StringType,true),
StructField("lll",StringType,true)))
def reorderSchema: StructType => StructType = schema => StructType(schema.sortBy(_.name))
var newSchema = reorderSchema(schema)
edited Mar 13 at 5:59
Prakash Thete
2,4201523
answered Mar 7 at 6:09
deodeo
32127
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55036314%2fhow-to-order-sort-the-structtype-based-on-the-field-names-in-java-spark-2-1-1%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In java,
new StructType(Stream.of(schema.fields())
.sorted(Comparator.comparing(StructField::name))
.collect(Collectors.toList()).toArray(new StructField[schema.fields().length]))
In scala
You can do something like,
var schema = StructType(Seq(StructField("zzz",StringType,true),
StructField("kkk",StringType,true),
StructField("aaa",StringType,true),
StructField("lll",StringType,true)))
def reorderSchema: StructType => StructType = schema => StructType(schema.sortBy(_.name))
var newSchema = reorderSchema(schema)
edited Mar 13 at 5:59
Prakash Thete
2,4201523
answered Mar 7 at 6:09
deodeo
32127
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
add a comment |
In java,
new StructType(Stream.of(schema.fields())
.sorted(Comparator.comparing(StructField::name))
.collect(Collectors.toList()).toArray(new StructField[schema.fields().length]))
In scala
You can do something like,
var schema = StructType(Seq(StructField("zzz",StringType,true),
StructField("kkk",StringType,true),
StructField("aaa",StringType,true),
StructField("lll",StringType,true)))
def reorderSchema: StructType => StructType = schema => StructType(schema.sortBy(_.name))
var newSchema = reorderSchema(schema)
edited Mar 13 at 5:59
Prakash Thete
2,4201523
answered Mar 7 at 6:09
deodeo
32127
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
add a comment |
In java,
new StructType(Stream.of(schema.fields())
.sorted(Comparator.comparing(StructField::name))
.collect(Collectors.toList()).toArray(new StructField[schema.fields().length]))
In scala
You can do something like,
var schema = StructType(Seq(StructField("zzz",StringType,true),
StructField("kkk",StringType,true),
StructField("aaa",StringType,true),
StructField("lll",StringType,true)))
def reorderSchema: StructType => StructType = schema => StructType(schema.sortBy(_.name))
var newSchema = reorderSchema(schema)
edited Mar 13 at 5:59
Prakash Thete
2,4201523
answered Mar 7 at 6:09
deodeo
32127
In java,
new StructType(Stream.of(schema.fields())
.sorted(Comparator.comparing(StructField::name))
.collect(Collectors.toList()).toArray(new StructField[schema.fields().length]))
In scala
You can do something like,
var schema = StructType(Seq(StructField("zzz",StringType,true),
StructField("kkk",StringType,true),
StructField("aaa",StringType,true),
StructField("lll",StringType,true)))
def reorderSchema: StructType => StructType = schema => StructType(schema.sortBy(_.name))
var newSchema = reorderSchema(schema)
edited Mar 13 at 5:59
Prakash Thete
2,4201523
answered Mar 7 at 6:09
deodeo
32127
edited Mar 13 at 5:59
Prakash Thete
2,4201523
edited Mar 13 at 5:59
Prakash Thete
2,4201523
edited Mar 13 at 5:59
Prakash Thete
2,4201523
2,4201523
answered Mar 7 at 6:09
deodeo
32127
answered Mar 7 at 6:09
deodeo
32127
answered Mar 7 at 6:09
deodeo
32127
32127
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
add a comment |
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
As i am using java, the mentioned answer is not working. we should pass the two parameters for sortBy(function f, order)
– svk 041994
Mar 7 at 6:22
updated the answer to work with java
– deo
Mar 7 at 7:01
updated the answer to work with java
– deo
Mar 7 at 7:01
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55036314%2fhow-to-order-sort-the-structtype-based-on-the-field-names-in-java-spark-2-1-1%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
