sqoop export error in oozie - set_ugi() not successful Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara Data science time! April 2019 and salary with experience The Ask Question Wizard is Live!Set variables in hive scripts using command lineError in sqoop action in oozie while fetching data from teradata to hiveoozie error while launching pig jobgetting error in oozie jobSqoop jobs are failing with java.lang.IllegalArgumentException: begin > end in range (begin, end): (1470951003366, 1470951003021)Sqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory OptionsMy Inputformat on hive cannot find when calling MapReduceWhile running jobs on hadoop,got the following errorsqoop export to postgresql is ERRORHow to run Spark program in Oozie?

France's Public Holidays' Puzzle

How to translate "red flag" into Spanish?

With indentation set to `0em`, when using a line break, there is still an indentation of a size of a space

Why didn't the Space Shuttle bounce back into space as many times as possible so as to lose a lot of kinetic energy up there?

Can you stand up from being prone using Skirmisher outside of your turn?

"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"

Why did C use the -> operator instead of reusing the . operator?

Is Electric Central Heating worth it if using Solar Panels?

Does Feeblemind produce an ongoing magical effect that can be dispelled?

Are these square matrices always diagonalisable?

Is Diceware more secure than a long passphrase?

Expansion//Explosion and Siren Stormtamer

Mistake in years of experience in resume?

What is the best way to deal with NPC-NPC combat?

What is the ongoing value of the Kanban board to the developers as opposed to management

Could Neutrino technically as side-effect, incentivize centralization of the bitcoin network?

Implementing 3DES algorithm in Java: is my code secure?

"Whatever a Russian does, they end up making the Kalashnikov gun"? Are there any similar proverbs in English?

How to use @AuraEnabled base class method in Lightning Component?

Are all CP/M-80 implementations binary compatible?

What ability score does a Hexblade's Pact Weapon use for attack and damage when wielded by another character?

How do I check if a string is entirely made of the same substring?

Why isn't everyone flabbergasted about Bran's "gift"?

As an international instructor, should I openly talk about my accent?



sqoop export error in oozie - set_ugi() not successful



Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
Data science time! April 2019 and salary with experience
The Ask Question Wizard is Live!Set variables in hive scripts using command lineError in sqoop action in oozie while fetching data from teradata to hiveoozie error while launching pig jobgetting error in oozie jobSqoop jobs are failing with java.lang.IllegalArgumentException: begin > end in range (begin, end): (1470951003366, 1470951003021)Sqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory OptionsMy Inputformat on hive cannot find when calling MapReduceWhile running jobs on hadoop,got the following errorsqoop export to postgresql is ERRORHow to run Spark program in Oozie?



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








0















I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



Logs:



2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
 at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
 at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
 at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
 at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
 at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
 at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
 at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
 at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
 at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
 at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
 at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
 at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
 at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
 at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
org.apache.thrift.transport.TTransportException
 at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
 at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
 at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
 at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
 at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
 at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
java.net.SocketException: Socket closed
 at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
 at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
 at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
 at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
 at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
 at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





hive hdfs sqoop oozie







share|improve this question






























    0















    I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



    I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



    Logs:



    2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
    2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
    2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
    org.apache.thrift.transport.TTransportException
     at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
     at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
     at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
     at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
     at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
     at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
     at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
     at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
     at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
     at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
     at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
     at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
     at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
     at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
    2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
    2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
    org.apache.thrift.transport.TTransportException
     at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
     at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
     at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
     at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
     at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
    2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
    2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
    java.net.SocketException: Socket closed
     at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
     at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
     at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
     at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
     at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
     at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
     at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





    hive hdfs sqoop oozie







    share|improve this question


























      0












      0








      0








      I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



      I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



      Logs:



      2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
      2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
      2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
       at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
       at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
       at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
       at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
       at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
       at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
       at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
       at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
      2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
      java.net.SocketException: Socket closed
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
       at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
       at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
       at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





      hive hdfs sqoop oozie







      share|improve this question
















      I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



      I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



      Logs:



      2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
      2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
      2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
       at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
       at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
       at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
       at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
       at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
       at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
       at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
       at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
      2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
      java.net.SocketException: Socket closed
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
       at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
       at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
       at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





      hive hdfs sqoop oozie




      hive hdfs sqoop oozie






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 9 at 8:08









      Mayank Patel

      2,96172746




      2,96172746










      asked Mar 9 at 6:23









      vjrock99vjrock99

      113




      113






















          0






          active

          oldest

          votes












          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55074641%2fsqoop-export-error-in-oozie-set-ugi-not-successful%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55074641%2fsqoop-export-error-in-oozie-set-ugi-not-successful%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          Save data to MySQL database using ExtJS and PHP [closed]2019 Community Moderator ElectionHow can I prevent SQL injection in PHP?Which MySQL data type to use for storing boolean valuesPHP: Delete an element from an arrayHow do I connect to a MySQL Database in Python?Should I use the datetime or timestamp data type in MySQL?How to get a list of MySQL user accountsHow Do You Parse and Process HTML/XML in PHP?Reference — What does this symbol mean in PHP?How does PHP 'foreach' actually work?Why shouldn't I use mysql_* functions in PHP?

          Image
          Does this AnyDice function accurately calculate the number of ogres you make unconcious with three 4th-level castings of Sleep? Making a sword in the stone, in a medieval world without magic Identifying the interval from A♭ to D♯ Why is Vishnu's skin colour mentioned as both white and blue in the Vishnu-sahasranaama? SQL Server Primary Login Restrictions Have researchers managed to "reverse time"? If so, what does that mean for physics? Why does Deadpool say "You're welcome, Canada," after shooting Ryan Reynolds in the end credits? Check this translation of Amores 1.3.26 Why did it take so long to abandon sail after steamships were demonstrated? Plot a function of two variables equal 0 2D counterpart of std::array in C++17 Could the Saturn V actually have launched astronauts around Venus? Why is "das Weib" grammatically neuter? How to make healing in an exploration game interesting Life insurance that covers only simulta
          Read more

          Compiling GNU Global with universal-ctags support Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Data science time! April 2019 and salary with experience The Ask Question Wizard is Live!Tags for Emacs: Relationship between etags, ebrowse, cscope, GNU Global and exuberant ctagsVim and Ctags tips and trickscscope or ctags why choose one over the other?scons and ctagsctags cannot open option file “.ctags”Adding tag scopes in universal-ctagsShould I use Universal-ctags?Universal ctags on WindowsHow do I install GNU Global with universal ctags support using Homebrew?Universal ctags with emacsHow to highlight ctags generated by Universal Ctags in Vim?

          Image
          How to ask rejected full-time candidates to apply to teach individual courses? Why these surprising proportionalities of integrals involving odd zeta values? A German immigrant ancestor has a "Registration Affidavit of Alien Enemy" on file. What does that mean exactly? Why did Israel vote against lifting the American embargo on Cuba? Unix AIX passing variable and arguments to expect and spawn Kepler's 3rd law: ratios don't fit data Why does my GNOME settings mention "Moto C Plus"? Continue tikz picture on next page What documents does someone with a long-term visa need to travel to another Schengen country? “Since the train was delayed for more than an hour, passengers were given a full refund.” – Why is there no article before “passengers”? How to create a command for the "strange m" symbol in latex? How to get a single big right brace? Is the Mordenkainen's Sword spell underpowered? Why did Bronn offer to be Tyrion L
          Read more

          Add ONERROR event to image from jsp tldHow to add an image to a JPanel?Saving image from PHP URLHTML img scalingCheck if an image is loaded (no errors) with jQueryHow to force an <img> to take up width, even if the image is not loadedHow do I populate hidden form field with a value set in Spring ControllerStyling Raw elements Generated from JSP tagds with Jquery MobileLimit resizing of images with explicitly set width and height attributeserror TLD use in a jsp fileJsp tld files cannot be resolved

          Image
          At which point does a character regain all their Hit Dice? Why are on-board computers allowed to change controls without notifying the pilots? How do I keep an essay about "feeling flat" from feeling flat? Time travel short story where a man arrives in the late 19th century in a time machine and then sends the machine back into the past How to be diplomatic in refusing to write code that breaches the privacy of our users Understanding "audieritis" in Psalm 94 Greatest common substring Go Pregnant or Go Home Why did Kant, Hegel, and Adorno leave some words and phrases in the Greek alphabet? What to do with wrong results in talks? Failed to fetch jessie backports repository Increase performance creating Mandelbrot set in python How can I replace every global instance of "x[2]" with "x_2" What are the ramifications of creating a homebrew world without an Astral Plane? Are there any comparative studies done between Ashtava
          Read more