Saving REST API JSON in RDBMS Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) The Ask Question Wizard is Live! Data science time! April 2019 and salary with experienceMapping JSON object to Hibernate entityHow do I format a Microsoft JSON date?Can comments be used in JSON?How can I pretty-print JSON in a shell script?What is the correct JSON content type?PUT vs. POST in RESTWhat exactly is RESTful programming?Why does Google prepend while(1); to their JSON responses?How can I pretty-print JSON using JavaScript?Parse JSON in JavaScript?How do I POST JSON data with Curl from a terminal/commandline to Test Spring REST?

If I can make up priors, why can't I make up posteriors?

Windows 10: How to Lock (not sleep) laptop on lid close?

What's the difference between (size_t)-1 and ~0?

Single author papers against my advisor's will?

Does the STL have a way to apply a function before calling less than?

Working around an AWS network ACL rule limit

Complexity of many constant time steps with occasional logarithmic steps

What LEGO pieces have "real-world" functionality?

What would be Julian Assange's expected punishment, on the current English criminal law?

If A makes B more likely then B makes A more likely"

What was Bilhah and Zilpah's ancestry?

What computer would be fastest for Mathematica Home Edition?

Stars Make Stars

I'm having difficulty getting my players to do stuff in a sandbox campaign

How can I protect witches in combat who wear limited clothing?

Is there folklore associating late breastfeeding with low intelligence and/or gullibility?

What did Darwin mean by 'squib' here?

Classification of bundles, Postnikov towers, obstruction theory, local coefficients

Problem when applying foreach loop

Statistical model of ligand substitution

How to colour the US map with Yellow, Green, Red and Blue to minimize the number of states with the colour of Green

What to do with post with dry rot?

Two different pronunciation of "понял"

What do you call a plan that's an alternative plan in case your initial plan fails?



Saving REST API JSON in RDBMS



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)
The Ask Question Wizard is Live!
Data science time! April 2019 and salary with experienceMapping JSON object to Hibernate entityHow do I format a Microsoft JSON date?Can comments be used in JSON?How can I pretty-print JSON in a shell script?What is the correct JSON content type?PUT vs. POST in RESTWhat exactly is RESTful programming?Why does Google prepend while(1); to their JSON responses?How can I pretty-print JSON using JavaScript?Parse JSON in JavaScript?How do I POST JSON data with Curl from a terminal/commandline to Test Spring REST?



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








1















There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.



DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.



I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.



EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.



Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity



Is there any better way to do this, or my planed approach is good for start?










share|improve this question
























  • May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

    – BigMike
    Mar 8 at 14:59






  • 1





    pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

    – Raymond Nijland
    Mar 8 at 15:01


















1















There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.



DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.



I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.



EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.



Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity



Is there any better way to do this, or my planed approach is good for start?










share|improve this question
























  • May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

    – BigMike
    Mar 8 at 14:59






  • 1





    pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

    – Raymond Nijland
    Mar 8 at 15:01














1












1








1








There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.



DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.



I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.



EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.



Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity



Is there any better way to do this, or my planed approach is good for start?










share|improve this question
















There is WebApp with REST API. I have to consume API, which is in JSON format, then save it in RDBMS. JSON is basically list of properties and sub-properties. I have to map them to the DB columns.



DB Schema is yet to be created whatever I think is suitable. What I am looking for is the best way to map JSON to Java DTO and then DTO to Entity. I was planning to use JACKSON to map JSON to DTO. I might or might not map all of the properties. Then in the next step I was planning of writing manual mappers who will map DTO to Entity.



I have to use RDBMS (MySQL) instead of NoSQL because other tools which will read data from DB are SQL Query based.



EDIT: Commercial reporting tools who are supposed to use RDBMS have some limited SQL Query capability, can't do any advanced SQL or business logic there. So I can't save JSON as string and then tokenise it later in reporting tools.



Any idea on this? I know it might be done directly JSON to Entity, but there are advice not to do such as in this advice: Mapping JSON object to Hibernate entity



Is there any better way to do this, or my planed approach is good for start?







java mysql json rest jackson






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 8 at 15:05







Nenad Bulatovic

















asked Mar 8 at 14:50









Nenad BulatovicNenad Bulatovic

4,69775983




4,69775983












  • May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

    – BigMike
    Mar 8 at 14:59






  • 1





    pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

    – Raymond Nijland
    Mar 8 at 15:01


















  • May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

    – BigMike
    Mar 8 at 14:59






  • 1





    pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

    – Raymond Nijland
    Mar 8 at 15:01

















May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

– BigMike
Mar 8 at 14:59





May sounds painfull but I would save the entire JSON string in a clob like column, mapping directly to proper colums just the important properties (those used for querying/sorting)

– BigMike
Mar 8 at 14:59




1




1





pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

– Raymond Nijland
Mar 8 at 15:01






pretty sure @BigMike is talking about Generated Columns here.. You use generated columns to parse out the JSON information which you need and you can index it.

– Raymond Nijland
Mar 8 at 15:01













2 Answers
2






active

oldest

votes


















1














Only one POJO model



I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.



You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.



Two POJO models



In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.



Collections and one POJO model



There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.



From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.






share|improve this answer






























    1














    In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).



    If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.



    But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.



    In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.






    share|improve this answer























      Your Answer






      StackExchange.ifUsing("editor", function ()
      StackExchange.using("externalEditor", function ()
      StackExchange.using("snippets", function ()
      StackExchange.snippets.init();
      );
      );
      , "code-snippets");

      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "1"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55065673%2fsaving-rest-api-json-in-rdbms%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      1














      Only one POJO model



      I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.



      You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.



      Two POJO models



      In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.



      Collections and one POJO model



      There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.



      From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.






      share|improve this answer



























        1














        Only one POJO model



        I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.



        You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.



        Two POJO models



        In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.



        Collections and one POJO model



        There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.



        From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.






        share|improve this answer

























          1












          1








          1







          Only one POJO model



          I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.



          You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.



          Two POJO models



          In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.



          Collections and one POJO model



          There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.



          From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.






          share|improve this answer













          Only one POJO model



          I agree that using the same POJO structure for Hibernate and Jackson libraries is not the best option, generally speaking. But it also depends from given scenarios you need to implement. In your case you need to load data from REST API and store result in DB. So, basically you will be invoking INSERT queries in 90% of cases. So, you can keep your POJO structure very simple without worrying about any lazy-loading problems. In this scenario using same POJO structure is not so bad. Linked question/answer says only about problems with reading and exposing DB structure by REST API which is not a case in this question.



          You need to also find out how different DB model will be from JSON model. If they are similar and you can say 1:1 there is no need to create extra layer.



          Two POJO models



          In this scenario you need to create two POJO models: one for handling JSON deserialisation and one for working with DB via ORM. In this scenario the biggest problem is to map these two models. Also, changes in REST API propagates over every layer. You can use of course map libraries such as Dozer, Orika, MapStruct or others but there is always a need to maintain this layer. From other side this is very safe solution because you can control mapping to JSON and to DB separately and keep different structures without many annotations or custom deserialisers, adapters, etc. From other side, if these two models are really different this solution is the best.



          Collections and one POJO model



          There is also a third option which I see and it is using Java collection library for handling JSON payloads. JSON Object fits to Map<String, Object> and JSON array fits to List<Object>. Jackson automatically take care about choosing right types and you need to only map these collections to POJO model used on ORM side. This solution removes need to keep two models but it complicates mapping layer and makes it fuzzy.



          From information you gave I would start from first option. In case, things are going to complicate generating extra model and mapping layer with help of some libraries should not be a big problem.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Mar 8 at 23:20









          Michał ZioberMichał Ziober

          17.7k1271109




          17.7k1271109























              1














              In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).



              If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.



              But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.



              In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.






              share|improve this answer



























                1














                In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).



                If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.



                But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.



                In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.






                share|improve this answer

























                  1












                  1








                  1







                  In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).



                  If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.



                  But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.



                  In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.






                  share|improve this answer













                  In general the approach is correct. But details depend on what frameworks/ORMs you're about to use (if any).



                  If you want to use JPA/Hibernate for persistence - you'd need an Entity class(es), which should never be used as DTOs. If you'll have some custom persistence you could probably skip entity classes altogether and just use DTO to populate query parameters.



                  But simplest and most common way to implement such a solution would be via using Spring framework which would automatically map your JSON to DTO using Jackson, and Spring Data JPA for persistence.



                  In this case the only work remaining is mapping DTO to Entity. This can be implemented manually, but can also be done using framework that would generate mappers - like MapStruct.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Mar 8 at 23:58









                  mvmnmvmn

                  1,8091524




                  1,8091524



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Stack Overflow!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55065673%2fsaving-rest-api-json-in-rdbms%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      AWS Lex not identifying response if by a variable The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern) The Ask Question Wizard is Live! Data science time! April 2019 and salary with experienceEnforcing custom enumeration in AWS LEX for slot valuesHow to give response based on user response in Amazon Lex?Intercepting AWS Lambda Response to a AWS Lex QueryLex chat bot error: Reached second execution of fulfillment lambda on the same utteranceamazon lex showing invalid responseLambda response send back to Lex slot?Response card in Amazon lexAmazon Lex - Lambda response return HTML to botHow can I solve 424 (Failed Dependency) (python) obtained from Amazon lex?

                      Алба-Юлія

                      Захаров Федір Захарович