Wednesday, January 3, 2018

Using Open Source R-Studio Server connecting to Kerberos-enabled Hadoop

Step 1: Add line "SPARK_HOME=${SPARK_HOME-'/opt/cloudera/parcels/CDH/lib/spark/'}" to end of file "/usr/lib64/R/etc/Renviron"

Step 2: Connect to Spark using sparklyr inside R-Studio Server

> install.packages("sparklyr")
> library(sparklyr)
> readRenviron("/usr/lib64/R/etc/Renviron")
> sc <- spark_connect(master = "yarn-client",version = "1.6.0", config = list
(default = list(spark.yarn.keytab = "/home/donghua/donghua.keytab", spark.yarn.principal = "donghua@DBAGLOBE.COM")))
> sc
$master
[1] "yarn-client"

$method
[1] "shell"

$app_name
[1] "sparklyr"

$config
$config$default
$config$default$spark.yarn.keytab
[1] "/home/donghua/donghua.keytab"

$config$default$spark.yarn.principal
[1] "donghua@DBAGLOBE.COM"



$spark_home
[1] "/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/lib/spark"

$backend
A connection with                               
description "->localhost:46015"
class       "sockconn"         
mode        "wb"               
text        "binary"           
opened      "opened"           
can read    "yes"              
can write   "yes"              

$monitor
A connection with                              
description "->localhost:8880"
class       "sockconn"        
mode        "rb"              
text        "binary"          
opened      "opened"          
can read    "yes"             
can write   "yes"             

$output_file
[1] "/tmp/RtmpXWaXfE/file7af1ca61a03_spark.log"

$spark_context
<jobj[6]>
  class org.apache.spark.SparkContext
  org.apache.spark.SparkContext@355d7d99

$java_context
<jobj[7]>
  class org.apache.spark.api.java.JavaSparkContext
  org.apache.spark.api.java.JavaSparkContext@ef616c5

attr(,"class")
[1] "spark_connection"       "spark_shell_connection" "DBIConnection"   


image

> library(DBI)
> iotdatademo <- dbGetQuery(sc, 'Select * from default.iotdatademo limit 10')
> iotdatademo

image

Reference URL: https://medium.com/@bkvarda/sparklyr-r-interface-for-spark-and-kerberos-on-cloudera-80abf5f6b4ad

86 comments:

  1. To inquire about the enormous measure of information we need an expert and all around experienced Data Scientist, to end up one of the specialists among the Data Scientist swarm, you should upgrade your range of abilities in huge information in a propelled manner. ExcelR Data Science Courses

    ReplyDelete
  2. I am genuinely thankful to the holder of this web page who has shared this wonderful paragraph at at this place
    360digitmg

    ReplyDelete
  3. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.Best data science courses in hyerabad

    ReplyDelete
  4. This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again. photographe

    ReplyDelete
  5. I am very happy to discover your post as it will become on top in my collection of favorite blogs to visit. WordPress Developer Brisbane

    ReplyDelete
  6. Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.
    토토사이트

    ReplyDelete
  7. I like this post,And I guess that they having fun to read this post,they shall take a good site to make a information,thanks for sharing it to me.
    파워볼사이트

    ReplyDelete
  8. Superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place 스포츠토토사이트

    ReplyDelete
  9. The registered guarantor at reliable. Always fighting! I'll come often. Great work! 보증업체

    ReplyDelete
  10. I can grow up with. Please continue to do well and always do well 먹튀폴리스 I'm surprised there's such a wonderful article

    ReplyDelete
  11. I live in a different country than you This post is invaluable 카지노커뮤니티

    ReplyDelete
  12. "It’s amazing in support of me to have a site, which is good in favor of my know-how. thanks admin" 메이저놀이터

    ReplyDelete
  13. understand everything, this is really fun and enjoyable for me. Thank you guys for sharing this. Click here, 토토사이트검증

    ReplyDelete
  14. Good website! I truly love how it is easy on my eyes it is. I am wondering how I might be notified whenever a new post has been made. I have subscribed to your RSS which may do the trick? Have a great day! 먹튀사이트

    ReplyDelete
  15. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. i99pro

    ReplyDelete
  16. Great articles and great layout. Your blog post deserves all of the positive feedback it’s been getting. 안전놀이터

    ReplyDelete
  17. You have done a great job on this article. It’s very readable and highly intelligent. You have even managed to make it understandable and easy to read. You have some real writing talent. Thank you. cheap hosting in indiaonohosting

    ReplyDelete
  18. An fascinating discussion is value comment. I think that it is best to write extra on this matter, it won’t be a taboo topic however generally people are not enough to talk on such topics. To the next. Cheers 안전놀이터

    ReplyDelete
  19. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. บาคาร่า

    ReplyDelete
  20. I am thankful to you for sharing this plethora of useful information. I found this resource utmost beneficial for me. Thanks a lot for hard work. 먹튀검증

    ReplyDelete
  21. Thank you so much as you have been willing to share information with us. We will forever admire all you have done here because you have made my work as easy as ABC. best digital perm

    ReplyDelete
  22. I cannot wait to dig deep and kickoff utilizing resources that I received from you. Your exuberance is refreshing. เว็บพนันบอลดีที่สุด

    ReplyDelete
  23. Everything has its value. Thanks for sharing this informative information with us. GOOD works! here

    ReplyDelete
  24. It is my first visit to your blog, and I am very impressed with the articles that you serve. Give adequate knowledge for me. Thank you for sharing useful material. I will be back for the more great post. 안전놀이터

    ReplyDelete
  25. Good website! I truly love how it is easy on my eyes it is. I am wondering how I might be notified whenever a new post has been made. I have subscribed to your RSS which may do the trick? Have a great day! movie4me

    ReplyDelete
  26. Pretty good post. I have just stumbled upon your blog and enjoyed reading your blog posts very much. I am looking for new posts to get more precious info. Big thanks for the useful info. subcene

    ReplyDelete
  27. I really like your writing style, great information, thankyou for posting. matka tips

    ReplyDelete
  28. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. matka

    ReplyDelete
  29. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. Watergardens at Canberra

    ReplyDelete
  30. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. Pollen Collection Price

    ReplyDelete
  31. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. 슬롯사이트

    ReplyDelete
  32. Hi there! Nice stuff, do keep me posted when you post again something like this! bolly4u

    ReplyDelete
  33. That is the excellent mindset, nonetheless is just not help to make every sence whatsoever preaching about that mather. Virtually any method many thanks in addition to i had endeavor to promote your own article in to delicius nevertheless it is apparently a dilemma using your information sites can you please recheck the idea. thanks once more. 안전놀이터

    ReplyDelete
  34. Thank you for such a well written article. It’s full of insightful information and entertaining descriptions. Your point of view is the best among many. 먹튀검증

    ReplyDelete
  35. This is my first visit to your web journal! We are a group of volunteers and new activities in the same specialty. Website gave us helpful data to work. 무료릴게임

    ReplyDelete
  36. Thank you so much as you have been willing to share information with us. We will forever admire all you have done here because you have made my work as easy as ABC. https://biolinky.co/togelsgp

    ReplyDelete
  37. You have outdone yourself this time. It is probably the best, most short step by step guide that I have ever seen. magicien mentaliste Lyon

    ReplyDelete
  38. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. สล็อต

    ReplyDelete
  39. Great article Lot's of information to Read...Great Man Keep Posting and update to People..Thanks 메이저사이트

    ReplyDelete
  40. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. ceme deposit pulsa

    ReplyDelete
  41. Someone Sometimes with visits your blog regularly and recommended it in my experience to read as well. The way of writing is excellent and also the content is top-notch. Thanks for that insight you provide the readers! 389sports

    ReplyDelete
  42. Cool stuff you have got and you keep update all of us. sattamataka143

    ReplyDelete
  43. Nice post. I was checking constantly this blog and I’m impressed! Extremely useful info specially the last part I care for such information a lot. I was seeking this certain info for a long time. Thank you and good luck. Alliance Broadband

    ReplyDelete
  44. Join Public Mobile now with Public Mobile Promo Code E3LRK5 and get $10 off your cell phone bill using a Public Mobile Referral Code exclusively on Newsrooms. Newsrooms

    ReplyDelete
  45. Contact a Toronto lawyer for a free consultation by phone, text, WhatsApp, WeChat, Zoom or E-mail using Instant Lawyer Instant Lawyer

    ReplyDelete
  46. The post is written in very a good manner and it contains many useful information for me. buy packwoods white runtz for sale

    ReplyDelete
  47. i am always looking for some free stuffs over the internet. there are also some companies which gives free samples. order packwoods marathon Gummies

    ReplyDelete
  48. Keep up the good work , I read few posts on this web site and I conceive that your blog is very interesting and has sets of fantastic information. best merchant services company to work for

    ReplyDelete
  49. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. how to fix dull hair

    ReplyDelete
  50. An interesting dialogue is price comment. I feel that it is best to write more on this matter, it may not be a taboo topic however usually individuals are not enough to talk on such topics. To the next. Cheers. joker388

    ReplyDelete
  51. An interesting dialogue is price comment. I feel that it is best to write more on this matter, it may not be a taboo topic however usually individuals are not enough to talk on such topics. To the next. Cheers. sbobet asia

    ReplyDelete
  52. Cool stuff you have got and you keep update all of us. academic transcription services

    ReplyDelete
  53. liposuction is the most common cosmetic surgical procedure performed because it works, and the results are dramatic, particularly for the right patients. But are you a good candidate for liposuction? go

    ReplyDelete
  54. Most patients want to have liposuction done by a specialist. A liposuction specialist is someone who performs this procedure almost daily. get more

    ReplyDelete
  55. Impressive web site, Distinguished feedback that I can tackle. Im moving forward and may apply to my current job as a pet sitter, which is very enjoyable, but I need to additional expand. Regards. 123movie

    ReplyDelete
  56. Hi there! Nice stuff, do keep me posted when you post again something like this! 안전놀이터

    ReplyDelete
  57. I appreciated your work very thanks 팔팔카

    ReplyDelete
  58. Great articles and great layout. Your blog post deserves all of the positive feedback it’s been getting. บาคาร่าออนไลน์

    ReplyDelete
  59. A great content material as well as great layout. Your website deserves all of the positive feedback it’s been getting. I will be back soon for further quality contents. 토토커뮤니티

    ReplyDelete
  60. I am unable to read articles online very often, but I’m glad I did today. This is very well written and your points are well-expressed. Please, don’t ever stop writing. 먹튀폴리스

    ReplyDelete
  61. I would like to thank you for the efforts you have made in writing this article. I am hoping the same best work from you in the future as well. Thanks... 바둑이

    ReplyDelete
  62. I read that Post and got it fine and informative. detectives privados barcelona

    ReplyDelete
  63. Thanks For sharing this Superb article.I use this Article to show my assignment in college.it is useful For me Great Work. 토토사이트

    ReplyDelete
  64. Nice post mate, keep up the great work, just shared this with my friendz 꽁머니 즉시환전

    ReplyDelete
  65. Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me. 꽁머니

    ReplyDelete
  66. Pretty good post. I have just stumbled upon your blog and enjoyed reading your blog posts very much. I am looking for new posts to get more precious info. Big thanks for the useful info. 토토사이트

    ReplyDelete
  67. Thanks For sharing this Superb article.I use this Article to show my assignment in college.it is useful For me Great Work. Lottery Sambad

    ReplyDelete
  68. Really a great addition. I have read this marvelous post. Thanks for sharing information about it. I really like that. Thanks so lot for your convene. 먹튀검증

    ReplyDelete
  69. Hey, I am so thrilled I found your blog, I am here now and could just like to say thank for a tremendous post and all round interesting website. Please do keep up the great work. I cannot be without visiting your blog again and again. 오피아트

    ReplyDelete
  70. I really loved reading your blog. It was very well authored and easy to undertand. Unlike additional blogs I have read which are really not tht good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he ejoyed it as well! 먹튀검증사이트

    ReplyDelete
  71. A betting site list to play online sports betting in Turkey here, visit immediately. antminer L7

    ReplyDelete
  72. I found this is an informative and interesting post so i think so it is very useful and knowledgeable. I would like to thank you for the efforts you have made in writing this article. If more people that write articles really concerned themselves with writing great content like you, more reader s would be interested in their writings. Thank you for caring about your content. You have a real talent for writing unique content. I like how you think and the way you express your views in this article. I am impressed by your writing style a lot. Thanks for making my experience more beautiful . I have recently started a blog, the info you provide on this site has helped me greatly. Thanks for all of your time & work. 오래된토토사이트

    ReplyDelete
  73. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value. Im glad to have found this post as its such an interesting one! I am always on the lookout for quality posts and articles so i suppose im lucky to have found this! I hope you will be adding more in the future.. Extremely helpful information specially the last part I care for such info a lot. I was seeking this particular information for a very long time. Thank you and good luck. Wow, cool post. I'd like to write like this too - taking time and real hard work to make a great article... but I put things off too much and never seem to get started. Thanks though. 토디즈

    ReplyDelete
  74. Very useful post. This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. Really its great article. Keep it up. Nice post! This is a very nice blog that I will definitively come back to more times this year! Thanks for informative post. I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks I think this is an informative post and it is very beneficial and knowledgeable. Therefore, I would like to thank you for the endeavors that you have made in writing this article. All the content is absolutely well-researched. Thanks 메이저공원

    ReplyDelete
  75. I’m excited to uncover this page. I need to to thank you for ones time for this particularly fantastic read!! I definitely really liked every part of it and i also have you saved to fav to look at new information in your site. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites! There's no doubt i would fully rate it after i read what is the idea about this article. You did a nice job.. 먹튀신고

    ReplyDelete
  76. I enjoy you because of your whole work on this blog. My daughter loves engaging in research and it’s really simple to grasp why. Most people know all of the compelling way you give vital items by means of the web blog and invigorate participation from the others on that subject matter while our own girl is always discovering a whole lot. Take pleasure in the remaining portion of the year. You’re carrying out a powerful job. Nice post. I learn some thing much harder on various blogs everyday. It will always be stimulating to learn to read content using their company writers and exercise something from their store. I’d choose to apply certain together with the content on my blog whether you don’t mind. Natually I’ll supply you with a link on your web weblog. Appreciate your sharing. 메이저놀이터검증

    ReplyDelete
  77. Thanks for a very interesting blog. What else may I get that kind of info written in such a perfect approach? I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info. this article is really one of the very best in the history of articles. I am a antique ’Article’ collector and I sometimes read some new articles if I find them interesting. And I found this one pretty fascinating and it should go into my collection. Very good work! Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future also 승인전화없는 토토꽁머니

    ReplyDelete
  78. I am really impressed that there is so much information about this subject that have been uncovered and you’ve done your best, with so much class. I was recommended this blog by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my difficulty. You’re wonderful! Thanks! Impressive web site, Distinguished feedback that I can tackle. Im moving forward and may apply to my current job as a pet sitter, which is very enjoyable, but I need to additional expand. Acknowledges for paper such a beneficial composition, I stumbled beside your blog besides decipher a limited announce. I want your technique of inscription.. 모두의토토

    ReplyDelete
  79. I’ve been surfing online more than three hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my opinion, if all webmasters and bloggers made good content as you did, the web will be a lot more useful than ever before. It is a completely interesting blog publish.I often visit your posts for my project's help about Diwali Bumper Lottery and your super writing capabilities genuinely go away me taken aback . I was suggested this website by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my trouble. You are wonderful! Thanks 토토디펜드

    ReplyDelete
  80. I am typically to running a blog and i really recognize your content. The article has actually peaks my interest. I am going to bookmark your website and keep checking for new information. I wish more authors of this type of content would take the time you did to research and write so well. I am very impressed with your vision and insight. Thanks for the post loaded with so much information. Stopping by your blog helped me to get what I was looking for. You've completed excellent work. Very good blog post.Much thanks again. Cool. 토토디펜드

    ReplyDelete
  81. first-rate data, valuable and first-rate layout, as proportion top stuff with exact thoughts and ideas, masses of great information and concept, each of which i want, way to offer one of these useful records here. Absolutely, this article is actually one of the very best inside the history of articles. I'm a antique ’article’ collector and that i on occasion examine some new articles if i locate them interesting. And i found this one quite charming and it should go into my series. Superb work! I regularly experience locating a internet site that offers useful info due to the fact i really like mastering interesting matters. Regularly instances while i've been to this web page i've really enjoyed the best article right here. An great weblog and i'm able to go back plenty greater for more precise content material… i am really impressed along with your writing competencies and additionally with the layout in your blog. Is that this a paid subject or did you modify it yourself? Anyway keep up the remarkable pleasant writing, it's miles rare to peer a top notch blog like this one these days. Super article, it turned into rather helpful! I in reality commenced on this and i am becoming extra acquainted with it higher! Cheers, hold doing tremendous ! That is very exciting content! I have thoroughly loved reading your points and feature come to the conclusion that you are right approximately a lot of them. You are exceptional. Amazing post i need to say and thank you for the information. Training is actually a sticky difficulty. However, continues to be most of the leading topics of our time. I respect your post and stay up for more. I really like this submit,and that i figure that they having a ton of a laugh to peruse this post,they may take a decent web page to make an data,thank you for sharing it to me. I appreciate this article for the well-researched content and exceptional wording. I were given so worried in this material that i couldn’t forestall analyzing. I am impressed along with your paintings and talent. Thank you so much. Superb information sharing .. I am very satisfied to examine this newsletter .. Thank you for giving us go through data. Exquisite great. I appreciate this publish 스포츠토토

    ReplyDelete
  82. Thanks for fantastic information I was looking for this information for my mission.
    play game for relax and win get money
    PG SLOT

    ReplyDelete
  83. having study this i thought it become very informative. I recognize you taking the effort and time to position this newsletter collectively. I once again discover myself spending way to a lot time both reading and commenting. But so what, it was nevertheless worth it! I without a doubt admire this publish. I’ve been searching everywhere for this! Thank goodness i discovered it on bing. You’ve made my day! Thank you again! I simply couldn’t leave your web web site prior to suggesting that i genuinely loved the standard facts someone offer in your traffic? Is gonna be returned regularly in order to check up on new posts 토토

    ReplyDelete
  84. Wow! Excellent stuff, just Excellent! UFABET I truly thank you for your content. Your article is a good article. I like your writing style, it's easy to understand and amazing.

    ReplyDelete