Saving REST API JSON in RDBMS Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) The Ask Question Wizard is Live! Data science time! April 2019 and salary with experienceMapping JSON object to Hibernate entityHow do I format a Microsoft JSON date?Can comments be used in JSON?How can I pretty-print JSON in a shell script?What is the correct JSON content type?PUT vs. POST in RESTWhat exactly is RESTful programming?Why does Google prepend while(1); to their JSON responses?How can I pretty-print JSON using JavaScript?Parse JSON in JavaScript?How do I POST JSON data with Curl from a terminal/commandline to Test Spring REST?
If I can make up priors, why can't I make up posteriors?
Windows 10: How to Lock (not sleep) laptop on lid close?
What's the difference between (size_t)-1 and ~0?
Single author papers against my advisor's will?
Does the STL have a way to apply a function before calling less than?
Working around an AWS network ACL rule limit
Complexity of many constant time steps with occasional logarithmic steps
What LEGO pieces have "real-world" functionality?
What would be Julian Assange's expected punishment, on the current English criminal law?
If A makes B more likely then B makes A more likely"
What was Bilhah and Zilpah's ancestry?
What computer would be fastest for Mathematica Home Edition?
Stars Make Stars
I'm having difficulty getting my players to do stuff in a sandbox campaign
How can I protect witches in combat who wear limited clothing?
Is there folklore associating late breastfeeding with low intelligence and/or gullibility?
What did Darwin mean by 'squib' here?
Classification of bundles, Postnikov towers, obstruction theory, local coefficients
Problem when applying foreach loop
Statistical model of ligand substitution
How to colour the US map with Yellow, Green, Red and Blue to minimize the number of states with the colour of Green
What to do with post with dry rot?
Two different pronunciation of "понял"
What do you call a plan that's an alternative plan in case your initial plan fails?
Saving REST API JSON in RDBMS
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
The Ask Question Wizard is Live!
Data science time! April 2019 and salary with experienceMapping JSON object to Hibernate entityHow do I format a Microsoft JSON date?Can comments be used in JSON?How can I pretty-print JSON in a shell script?What is the correct JSON content type?PUT vs. POST in RESTWhat exactly is RESTful programming?Why does Google prepend while(1); to their JSON responses?How can I pretty-print JSON using JavaScript?Parse JSON in JavaScript?How do I POST JSON data with Curl from a terminal/commandline to Test Spring REST?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.
DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.
I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.
EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.
Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity
Is there any better way to do this, or my planed approach is good for start?
java mysql json rest jackson
add a comment |
There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.
DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.
I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.
EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.
Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity
Is there any better way to do this, or my planed approach is good for start?
java mysql json rest jackson
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
1
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01
add a comment |
There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.
DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.
I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.
EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.
Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity
Is there any better way to do this, or my planed approach is good for start?
java mysql json rest jackson
There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.
DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.
I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.
EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.
Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity
Is there any better way to do this, or my planed approach is good for start?
java mysql json rest jackson
java mysql json rest jackson
edited Mar 8 at 15:05
Nenad Bulatovic
asked Mar 8 at 14:50
Nenad BulatovicNenad Bulatovic
4,69775983
4,69775983
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
1
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01
add a comment |
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
1
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
1
1
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01
add a comment |
2 Answers
2
active
oldest
votes
Only one POJO model
I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.
You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.
Two POJO models
In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.
Collections and one POJO model
There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.
From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.
add a comment |
In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).
If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.
But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.
In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55065673%2fsaving-rest-api-json-in-rdbms%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Only one POJO model
I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.
You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.
Two POJO models
In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.
Collections and one POJO model
There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.
From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.
add a comment |
Only one POJO model
I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.
You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.
Two POJO models
In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.
Collections and one POJO model
There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.
From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.
add a comment |
Only one POJO model
I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.
You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.
Two POJO models
In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.
Collections and one POJO model
There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.
From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.
Only one POJO model
I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.
You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.
Two POJO models
In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.
Collections and one POJO model
There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.
From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.
answered Mar 8 at 23:20
Michał ZioberMichał Ziober
17.7k1271109
17.7k1271109
add a comment |
add a comment |
In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).
If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.
But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.
In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.
add a comment |
In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).
If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.
But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.
In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.
add a comment |
In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).
If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.
But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.
In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.
In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).
If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.
But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.
In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.
answered Mar 8 at 23:58
mvmnmvmn
1,8091524
1,8091524
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55065673%2fsaving-rest-api-json-in-rdbms%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)
– BigMike
Mar 8 at 14:59
1
pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.
– Raymond Nijland
Mar 8 at 15:01