Wednesday, January 3, 2018

Using Open Source R-Studio Server connecting to Kerberos-enabled Hadoop

Step 1: Add line "SPARK_HOME=${SPARK_HOME-'/opt/cloudera/parcels/CDH/lib/spark/'}" to end of file "/usr/lib64/R/etc/Renviron"

Step 2: Connect to Spark using sparklyr inside R-Studio Server

> install.packages("sparklyr")
> library(sparklyr)
> readRenviron("/usr/lib64/R/etc/Renviron")
> sc <- spark_connect(master = "yarn-client",version = "1.6.0", config = list
(default = list(spark.yarn.keytab = "/home/donghua/donghua.keytab", spark.yarn.principal = "donghua@DBAGLOBE.COM")))
> sc
$master
[1] "yarn-client"

$method
[1] "shell"

$app_name
[1] "sparklyr"

$config
$config$default
$config$default$spark.yarn.keytab
[1] "/home/donghua/donghua.keytab"

$config$default$spark.yarn.principal
[1] "donghua@DBAGLOBE.COM"



$spark_home
[1] "/opt/cloudera/parcels/CDH-5.13.1-1.cdh5.13.1.p0.2/lib/spark"

$backend
A connection with                               
description "->localhost:46015"
class       "sockconn"         
mode        "wb"               
text        "binary"           
opened      "opened"           
can read    "yes"              
can write   "yes"              

$monitor
A connection with                              
description "->localhost:8880"
class       "sockconn"        
mode        "rb"              
text        "binary"          
opened      "opened"          
can read    "yes"             
can write   "yes"             

$output_file
[1] "/tmp/RtmpXWaXfE/file7af1ca61a03_spark.log"

$spark_context
<jobj[6]>
  class org.apache.spark.SparkContext
  org.apache.spark.SparkContext@355d7d99

$java_context
<jobj[7]>
  class org.apache.spark.api.java.JavaSparkContext
  org.apache.spark.api.java.JavaSparkContext@ef616c5

attr(,"class")
[1] "spark_connection"       "spark_shell_connection" "DBIConnection"   


image

> library(DBI)
> iotdatademo <- dbGetQuery(sc, 'Select * from default.iotdatademo limit 10')
> iotdatademo

image

Reference URL: https://medium.com/@bkvarda/sparklyr-r-interface-for-spark-and-kerberos-on-cloudera-80abf5f6b4ad

65 comments:

  1. To inquire about the enormous measure of information we need an expert and all around experienced Data Scientist, to end up one of the specialists among the Data Scientist swarm, you should upgrade your range of abilities in huge information in a propelled manner. ExcelR Data Science Courses

    ReplyDelete
  2. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.Best data science courses in hyerabad

    ReplyDelete
  3. This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again. photographe

    ReplyDelete
  4. I am very happy to discover your post as it will become on top in my collection of favorite blogs to visit. WordPress Developer Brisbane

    ReplyDelete
  5. Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.
    토토사이트

    ReplyDelete
  6. Superbly written article, if only all bloggers offered the same content as you, the internet would be a far better place 스포츠토토사이트

    ReplyDelete
  7. The registered guarantor at reliable. Always fighting! I'll come often. Great work! 보증업체

    ReplyDelete
  8. I can grow up with. Please continue to do well and always do well 먹튀폴리스 I'm surprised there's such a wonderful article

    ReplyDelete
  9. I live in a different country than you This post is invaluable 카지노커뮤니티

    ReplyDelete
  10. "It’s amazing in support of me to have a site, which is good in favor of my know-how. thanks admin" 메이저놀이터

    ReplyDelete
  11. understand everything, this is really fun and enjoyable for me. Thank you guys for sharing this. Click here, 토토사이트검증

    ReplyDelete
  12. Good website! I truly love how it is easy on my eyes it is. I am wondering how I might be notified whenever a new post has been made. I have subscribed to your RSS which may do the trick? Have a great day! 먹튀사이트

    ReplyDelete
  13. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. i99pro

    ReplyDelete
  14. Great articles and great layout. Your blog post deserves all of the positive feedback it’s been getting. 안전놀이터

    ReplyDelete
  15. You have done a great job on this article. It’s very readable and highly intelligent. You have even managed to make it understandable and easy to read. You have some real writing talent. Thank you. cheap hosting in indiaonohosting

    ReplyDelete
  16. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. บาคาร่า

    ReplyDelete
  17. Thank you so much as you have been willing to share information with us. We will forever admire all you have done here because you have made my work as easy as ABC. best digital perm

    ReplyDelete
  18. I cannot wait to dig deep and kickoff utilizing resources that I received from you. Your exuberance is refreshing. เว็บพนันบอลดีที่สุด

    ReplyDelete
  19. It is my first visit to your blog, and I am very impressed with the articles that you serve. Give adequate knowledge for me. Thank you for sharing useful material. I will be back for the more great post. 안전놀이터

    ReplyDelete
  20. Good website! I truly love how it is easy on my eyes it is. I am wondering how I might be notified whenever a new post has been made. I have subscribed to your RSS which may do the trick? Have a great day! movie4me

    ReplyDelete
  21. Pretty good post. I have just stumbled upon your blog and enjoyed reading your blog posts very much. I am looking for new posts to get more precious info. Big thanks for the useful info. subcene

    ReplyDelete
  22. I really like your writing style, great information, thankyou for posting. matka tips

    ReplyDelete
  23. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. matka

    ReplyDelete
  24. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. Watergardens at Canberra

    ReplyDelete
  25. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. Pollen Collection Price

    ReplyDelete
  26. Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work. 슬롯사이트

    ReplyDelete
  27. Hi there! Nice stuff, do keep me posted when you post again something like this! bolly4u

    ReplyDelete
  28. Thank you for such a well written article. It’s full of insightful information and entertaining descriptions. Your point of view is the best among many. 먹튀검증

    ReplyDelete
  29. This is my first visit to your web journal! We are a group of volunteers and new activities in the same specialty. Website gave us helpful data to work. 무료릴게임

    ReplyDelete
  30. You have outdone yourself this time. It is probably the best, most short step by step guide that I have ever seen. magicien mentaliste Lyon

    ReplyDelete
  31. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. สล็อต

    ReplyDelete
  32. Great article Lot's of information to Read...Great Man Keep Posting and update to People..Thanks 메이저사이트

    ReplyDelete
  33. Wonderful blog! I found it while surfing around on Yahoo News. Do you have any suggestions on how to get listed in Yahoo News? I’ve been trying for a while but I never seem to get there! Appreciate it. ceme deposit pulsa

    ReplyDelete
  34. Someone Sometimes with visits your blog regularly and recommended it in my experience to read as well. The way of writing is excellent and also the content is top-notch. Thanks for that insight you provide the readers! 389sports

    ReplyDelete
  35. Cool stuff you have got and you keep update all of us. sattamataka143

    ReplyDelete
  36. Nice post. I was checking constantly this blog and I’m impressed! Extremely useful info specially the last part I care for such information a lot. I was seeking this certain info for a long time. Thank you and good luck. Alliance Broadband

    ReplyDelete
  37. Join Public Mobile now with Public Mobile Promo Code E3LRK5 and get $10 off your cell phone bill using a Public Mobile Referral Code exclusively on Newsrooms. Newsrooms

    ReplyDelete
  38. Contact a Toronto lawyer for a free consultation by phone, text, WhatsApp, WeChat, Zoom or E-mail using Instant Lawyer Instant Lawyer

    ReplyDelete
  39. The post is written in very a good manner and it contains many useful information for me. buy packwoods white runtz for sale

    ReplyDelete
  40. i am always looking for some free stuffs over the internet. there are also some companies which gives free samples. order packwoods marathon Gummies

    ReplyDelete
  41. Keep up the good work , I read few posts on this web site and I conceive that your blog is very interesting and has sets of fantastic information. best merchant services company to work for

    ReplyDelete
  42. Great post, you have pointed out some fantastic points , I likewise think this s a very wonderful website. how to fix dull hair

    ReplyDelete
  43. An interesting dialogue is price comment. I feel that it is best to write more on this matter, it may not be a taboo topic however usually individuals are not enough to talk on such topics. To the next. Cheers. joker388

    ReplyDelete
  44. An interesting dialogue is price comment. I feel that it is best to write more on this matter, it may not be a taboo topic however usually individuals are not enough to talk on such topics. To the next. Cheers. sbobet asia

    ReplyDelete
  45. Cool stuff you have got and you keep update all of us. academic transcription services

    ReplyDelete
  46. liposuction is the most common cosmetic surgical procedure performed because it works, and the results are dramatic, particularly for the right patients. But are you a good candidate for liposuction? go

    ReplyDelete
  47. Most patients want to have liposuction done by a specialist. A liposuction specialist is someone who performs this procedure almost daily. get more

    ReplyDelete
  48. Impressive web site, Distinguished feedback that I can tackle. Im moving forward and may apply to my current job as a pet sitter, which is very enjoyable, but I need to additional expand. Regards. 123movie

    ReplyDelete
  49. I appreciated your work very thanks 팔팔카

    ReplyDelete
  50. Great articles and great layout. Your blog post deserves all of the positive feedback it’s been getting. บาคาร่าออนไลน์

    ReplyDelete
  51. I read that Post and got it fine and informative. detectives privados barcelona

    ReplyDelete
  52. Nice post mate, keep up the great work, just shared this with my friendz 꽁머니 즉시환전

    ReplyDelete
  53. Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me. 꽁머니

    ReplyDelete
  54. Thanks For sharing this Superb article.I use this Article to show my assignment in college.it is useful For me Great Work. Lottery Sambad

    ReplyDelete
  55. I really loved reading your blog. It was very well authored and easy to undertand. Unlike additional blogs I have read which are really not tht good. I also found your posts very interesting. In fact after reading, I had to go show it to my friend and he ejoyed it as well! 먹튀검증사이트

    ReplyDelete
  56. A betting site list to play online sports betting in Turkey here, visit immediately. antminer L7

    ReplyDelete
  57. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value. Im glad to have found this post as its such an interesting one! I am always on the lookout for quality posts and articles so i suppose im lucky to have found this! I hope you will be adding more in the future.. Extremely helpful information specially the last part I care for such info a lot. I was seeking this particular information for a very long time. Thank you and good luck. Wow, cool post. I'd like to write like this too - taking time and real hard work to make a great article... but I put things off too much and never seem to get started. Thanks though. 토디즈

    ReplyDelete
  58. I’m excited to uncover this page. I need to to thank you for ones time for this particularly fantastic read!! I definitely really liked every part of it and i also have you saved to fav to look at new information in your site. Excellent Blog! I would like to thank for the efforts you have made in writing this post. I am hoping the same best work from you in the future as well. I wanted to thank you for this websites! Thanks for sharing. Great websites! There's no doubt i would fully rate it after i read what is the idea about this article. You did a nice job.. 먹튀신고

    ReplyDelete
  59. I enjoy you because of your whole work on this blog. My daughter loves engaging in research and it’s really simple to grasp why. Most people know all of the compelling way you give vital items by means of the web blog and invigorate participation from the others on that subject matter while our own girl is always discovering a whole lot. Take pleasure in the remaining portion of the year. You’re carrying out a powerful job. Nice post. I learn some thing much harder on various blogs everyday. It will always be stimulating to learn to read content using their company writers and exercise something from their store. I’d choose to apply certain together with the content on my blog whether you don’t mind. Natually I’ll supply you with a link on your web weblog. Appreciate your sharing. 메이저놀이터검증

    ReplyDelete
  60. I am really impressed that there is so much information about this subject that have been uncovered and you’ve done your best, with so much class. I was recommended this blog by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my difficulty. You’re wonderful! Thanks! Impressive web site, Distinguished feedback that I can tackle. Im moving forward and may apply to my current job as a pet sitter, which is very enjoyable, but I need to additional expand. Acknowledges for paper such a beneficial composition, I stumbled beside your blog besides decipher a limited announce. I want your technique of inscription.. 모두의토토

    ReplyDelete
  61. I’ve been surfing online more than three hours today, yet I never found any interesting article like yours. It’s pretty worth enough for me. In my opinion, if all webmasters and bloggers made good content as you did, the web will be a lot more useful than ever before. It is a completely interesting blog publish.I often visit your posts for my project's help about Diwali Bumper Lottery and your super writing capabilities genuinely go away me taken aback . I was suggested this website by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my trouble. You are wonderful! Thanks 토토디펜드

    ReplyDelete
  62. I am typically to running a blog and i really recognize your content. The article has actually peaks my interest. I am going to bookmark your website and keep checking for new information. I wish more authors of this type of content would take the time you did to research and write so well. I am very impressed with your vision and insight. Thanks for the post loaded with so much information. Stopping by your blog helped me to get what I was looking for. You've completed excellent work. Very good blog post.Much thanks again. Cool. 토토디펜드

    ReplyDelete
  63. first-rate data, valuable and first-rate layout, as proportion top stuff with exact thoughts and ideas, masses of great information and concept, each of which i want, way to offer one of these useful records here. Absolutely, this article is actually one of the very best inside the history of articles. I'm a antique ’article’ collector and that i on occasion examine some new articles if i locate them interesting. And i found this one quite charming and it should go into my series. Superb work! I regularly experience locating a internet site that offers useful info due to the fact i really like mastering interesting matters. Regularly instances while i've been to this web page i've really enjoyed the best article right here. An great weblog and i'm able to go back plenty greater for more precise content material… i am really impressed along with your writing competencies and additionally with the layout in your blog. Is that this a paid subject or did you modify it yourself? Anyway keep up the remarkable pleasant writing, it's miles rare to peer a top notch blog like this one these days. Super article, it turned into rather helpful! I in reality commenced on this and i am becoming extra acquainted with it higher! Cheers, hold doing tremendous ! That is very exciting content! I have thoroughly loved reading your points and feature come to the conclusion that you are right approximately a lot of them. You are exceptional. Amazing post i need to say and thank you for the information. Training is actually a sticky difficulty. However, continues to be most of the leading topics of our time. I respect your post and stay up for more. I really like this submit,and that i figure that they having a ton of a laugh to peruse this post,they may take a decent web page to make an data,thank you for sharing it to me. I appreciate this article for the well-researched content and exceptional wording. I were given so worried in this material that i couldn’t forestall analyzing. I am impressed along with your paintings and talent. Thank you so much. Superb information sharing .. I am very satisfied to examine this newsletter .. Thank you for giving us go through data. Exquisite great. I appreciate this publish 스포츠토토

    ReplyDelete
  64. having study this i thought it become very informative. I recognize you taking the effort and time to position this newsletter collectively. I once again discover myself spending way to a lot time both reading and commenting. But so what, it was nevertheless worth it! I without a doubt admire this publish. I’ve been searching everywhere for this! Thank goodness i discovered it on bing. You’ve made my day! Thank you again! I simply couldn’t leave your web web site prior to suggesting that i genuinely loved the standard facts someone offer in your traffic? Is gonna be returned regularly in order to check up on new posts 토토

    ReplyDelete
  65. Wow! Excellent stuff, just Excellent! UFABET I truly thank you for your content. Your article is a good article. I like your writing style, it's easy to understand and amazing.

    ReplyDelete