sqoop export error in oozie - set_ugi() not successful Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara Data science time! April 2019 and salary with experience The Ask Question Wizard is Live!Set variables in hive scripts using command lineError in sqoop action in oozie while fetching data from teradata to hiveoozie error while launching pig jobgetting error in oozie jobSqoop jobs are failing with java.lang.IllegalArgumentException: begin > end in range (begin, end): (1470951003366, 1470951003021)Sqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory OptionsMy Inputformat on hive cannot find when calling MapReduceWhile running jobs on hadoop,got the following errorsqoop export to postgresql is ERRORHow to run Spark program in Oozie?

France's Public Holidays' Puzzle

How to translate "red flag" into Spanish?

With indentation set to `0em`, when using a line break, there is still an indentation of a size of a space

Why didn't the Space Shuttle bounce back into space as many times as possible so as to lose a lot of kinetic energy up there?

Can you stand up from being prone using Skirmisher outside of your turn?

"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"

Why did C use the -> operator instead of reusing the . operator?

Is Electric Central Heating worth it if using Solar Panels?

Does Feeblemind produce an ongoing magical effect that can be dispelled?

Are these square matrices always diagonalisable?

Is Diceware more secure than a long passphrase?

Expansion//Explosion and Siren Stormtamer

Mistake in years of experience in resume?

What is the best way to deal with NPC-NPC combat?

What is the ongoing value of the Kanban board to the developers as opposed to management

Could Neutrino technically as side-effect, incentivize centralization of the bitcoin network?

Implementing 3DES algorithm in Java: is my code secure?

"Whatever a Russian does, they end up making the Kalashnikov gun"? Are there any similar proverbs in English?

How to use @AuraEnabled base class method in Lightning Component?

Are all CP/M-80 implementations binary compatible?

What ability score does a Hexblade's Pact Weapon use for attack and damage when wielded by another character?

How do I check if a string is entirely made of the same substring?

Why isn't everyone flabbergasted about Bran's "gift"?

As an international instructor, should I openly talk about my accent?



sqoop export error in oozie - set_ugi() not successful



Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
Data science time! April 2019 and salary with experience
The Ask Question Wizard is Live!Set variables in hive scripts using command lineError in sqoop action in oozie while fetching data from teradata to hiveoozie error while launching pig jobgetting error in oozie jobSqoop jobs are failing with java.lang.IllegalArgumentException: begin > end in range (begin, end): (1470951003366, 1470951003021)Sqoop import error message ERROR tool.ImportTool: Import failed: ENOENT: No such file or directory OptionsMy Inputformat on hive cannot find when calling MapReduceWhile running jobs on hadoop,got the following errorsqoop export to postgresql is ERRORHow to run Spark program in Oozie?



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








0















I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



Logs:



2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
 at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
 at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
 at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
 at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
 at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
 at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
 at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
 at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
 at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
 at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
 at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
 at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
 at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
 at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
 at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
org.apache.thrift.transport.TTransportException
 at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
 at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
 at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
 at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
 at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
 at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
 at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
java.net.SocketException: Socket closed
 at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
 at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
 at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
 at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
 at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
 at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
 at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
 at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
 at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
 at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
 at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
 at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
 at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
 at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
 at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
 at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
 at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
 at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
 at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
 at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
 at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
 at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





hive hdfs sqoop oozie







share|improve this question






























    0















    I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



    I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



    Logs:



    2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
    2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
    2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
    org.apache.thrift.transport.TTransportException
     at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
     at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
     at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
     at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
     at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
     at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
     at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
     at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
     at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
     at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
     at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
     at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
     at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
     at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
    2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
    2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
    org.apache.thrift.transport.TTransportException
     at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
     at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
     at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
     at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
     at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
     at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
     at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
    2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
    2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
    java.net.SocketException: Socket closed
     at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
     at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
     at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
     at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
     at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
     at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
     at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
     at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
     at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
     at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
     at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
     at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
     at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
     at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
     at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
     at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
     at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
     at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:498)
     at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:422)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





    hive hdfs sqoop oozie







    share|improve this question


























      0












      0








      0








      I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



      I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



      Logs:



      2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
      2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
      2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
       at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
       at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
       at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
       at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
       at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
       at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
       at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
       at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
      2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
      java.net.SocketException: Socket closed
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
       at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
       at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
       at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





      hive hdfs sqoop oozie







      share|improve this question
















      I implemented sqoop export (using hcatalog tables) in oozie workflow but got this error. I also copied hive-site.xml in hdfs code directory from server (path /etc/hive/conf/) but that didn't help.Do i need to pass/add any special parameters in hive-site.xml ?



      I am passing hcat.metastore.uri and hcat.metastore.principal values in workflow.xml



      Logs:



      2019-03-08 23:23:03,838 [main] INFO hive.metastore - Trying to connect to metastore with URI thrift://xxxx:1000
      2019-03-08 23:23:03,875 [main] INFO hive.metastore - Opened a connection to metastore, current connections: 1
      2019-03-08 23:23:03,964 [main] WARN hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3827)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3813)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:490)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:252)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:187)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:303)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:227)
       at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:224)
       at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4767)
       at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
       at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
       at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
       at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
       at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
       at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4764)
       at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:224)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:200)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,971 [main] INFO hive.metastore - Connected to metastore.
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Got exception: org.apache.thrift.transport.TTransportException null
      org.apache.thrift.transport.TTransportException
       at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
       at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
       at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
       at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
       at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_databases(ThriftHiveMetastore.java:730)
       at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_databases(ThriftHiveMetastore.java:717)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1083)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:333)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:203)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      2019-03-08 23:23:03,982 [main] ERROR hive.log - Converting exception to MetaException
      2019-03-08 23:23:03,989 [main] WARN org.apache.thrift.transport.TIOStreamTransport - Error closing output stream.
      java.net.SocketException: Socket closed
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
       at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
       at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
       at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
       at org.apache.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
       at org.apache.thrift.transport.TSocket.close(TSocket.java:235)
       at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.close(HiveMetaStoreClient.java:554)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDown(HiveClientCache.java:369)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.tearDownIfUnused(HiveClientCache.java:359)
       at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.close(HiveClientCache.java:349)
       at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:206)
       at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:558)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104)
       at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
       at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
       at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:348)
       at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:426)
       at org.apache.sqoop.manager.OracleManager.exportTable(OracleManager.java:465)
       at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
       at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
       at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
       at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
       at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
       at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
       at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
       at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
       at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
       at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
       at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
       at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
       at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:422)
       at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
       at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)





      hive hdfs sqoop oozie




      hive hdfs sqoop oozie






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Mar 9 at 8:08









      Mayank Patel

      2,96172746




      2,96172746










      asked Mar 9 at 6:23









      vjrock99vjrock99

      113




      113






















          0






          active

          oldest

          votes












          Your Answer






          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "1"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55074641%2fsqoop-export-error-in-oozie-set-ugi-not-successful%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55074641%2fsqoop-export-error-in-oozie-set-ugi-not-successful%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          1928 у кіно

          Image
          Роки в кіно   1870-ті   1880-ті 1888  • 1889 1890-ті 1890  • 1891  • 1892  • 1893  • 1894 1895  • 1896  • 1897  • 1898  • 1899 1900-ті 1900  • 1901  • 1902  • 1903  • 1904 1905  • 1906  • 1907  • 1908  • 1909 1910-ті 1910  • 1911  • 1912  • 1913  • 1914 1915  • 1916  • 1917  • 1918  • 1919 1920-ті 1920  • 1921  • 1922  • 1923  • 1924 1925  • 1926  • 1927  • 1928  • 1929 1930-ті 1930  • 1931  • 1932  • 1933  • 1934 1935  • 1936  • 1937  • 1938  • 1939 1940-ві 1940  • 1941  • 1942  • 1943  • 1944 1945  • 1946  • 1947  • 1948  • 1949 1950-ті 1950  • 1951  • 1952  • 1953  • 1954 1955  • 1956  • 1957  • 1958  • 1959 1960-ті 1960  • 1961  • 1962  • 1963  • 1964 1965  • 1966  • 1967  • 1968  • 1969 1970-ті 1970  • 1971  • 1972  • 1973  • 1974 1975  • 1976  • 1977  • 1978  • 1979 1980-ті 1980  • 1981  • 1982  • 1983  • 1984 1985  • 1986  • 1987  • 1988  • 1989 1990-ті 1990  • 1991  • 1992  • 1993  • 1994 1995  • 1996  • 1997  • 1998  • 1999 2000-ні 2000  • 2001  • 2002  • 2003  • 2004 2005
          Read more

          Захаров Федір Захарович

          Image
          Федір Захарович Захаров Федір Захаров, 1986 рік Народження 5 вересня 1919 ( 1919-09-05 ) Олександрівське [d] , Смоленська губернія , Російська СФРР Смерть 29 вересня 1997 ( 1997-09-29 ) (78 років)   Ялта , Автономна Республіка Крим , Україна Громадянство  СРСР →  Україна Навчання Московське художньо-промислове училище імені М. І. Калініна (1935—1941) Московський художній інститут імені В. І. Сурикова (1943—1950) Діяльність художник Напрямок імпресіонізм Роки творчості 1950—1994 Покровитель А. Лентулов І. Чекмасов Г. Ряжський Вплив російський живопис кінця 19 — початку 20 століть Вплив на Бондар Іван Іванович Нагороди Премії Звання Захаров Федір Захарович у Вікісховищі ? У Вікіпедії є статті про інших людей з прізвищем Захаров. Федір Захарович Захаров ( 5 вересня 1919 ( 1919-09-05 ) , Олександрівське [d] , Російська СФРР  — 29 вересня 1997 ( 1997-09-29 ) , Ялта , Україна ) — радянський та український художник, Заслужений діяч мистецтв УРСР (1970), народний художник УРС
          Read more

          Ель Греко

          Image
          Ель Греко ісп. El Greco «Портрет старого» (Автопортрет Ель Греко), бл. 1595–1600, олія на полотні При народженні Доменікос Теотокопулос Народження 1541 ( 1541 ) Крит, Венеціанська Республіка (нині Греція) Смерть 7 квітня 1614 ( 1614-04-07 )   Толедо, Іспанія Поховання Монастир Санто Домінго де Сілос [d] Національність грек Громадянство  Іспанія Релігія Римо-католицька церква Жанр портретний живопис [d] , міфологічний живопис і релігійний живопис [d] Діяльність художник , скульптор , архітектор , митець Напрямок маньєризм Вплив Тіціан Твори Воскресіння [d] , Краєвид Толедо і Погребіння графа Оргаса [d] Автограф Ель Греко у Вікісховищі ? Ель Гре́ко (ісп. El Greco — грек, справжнє ім'я Доменікос Теотокопулос (грец. Δομήνικος Θεοτοκόπουλος ), *1541, Крит, Венеціанська Республіка — †7 квітня 1614, Толедо, Іспанія) — іспанський художник грецького походження. Один із представників Критської школи . Зміст 1 Походження і навчання 2 Ікони 3 Автопортрети 4 Італійський
          Read more