Sunday 24 June 2018

Spark working with Unstructured data


Introduction
In my previous article with Spark, we worked with structure and semi structure data source. Here in this article we are trying to work with unstructured data source.
Hope it will be interesting.

Case Study
We have a note book and we want to find number of work count in it.



Scala Code

val dfsFilename = "D:/spark/bin/examples/src/main/resources/notebook.txt"

val text = sc.textFile(dfsFilename)
val counts = text.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey(_+_)
counts.collect.foreach(println)

Output






Hope it will be interesting.

Posted by: MR. JOYDEEP DAS

12 comments:

  1. I love this blog . This is one of the best blog i ever seen. It's all about what i'm searching for. I love to read this blog again and again . Every time i enter this blog i get something new. This blog inspire me to write new blog. I write a blog name http://tutorialabc.com. It's about sql,c#,net etc

    ReplyDelete

  2. It's so nice article thank you for sharing a valuable content. SQL server dba Online Training Bangalore

    ReplyDelete
  3. Hello Author,
    while I visit your blog I found that your content is so clear which attracts user attention. thanks for sharing a worthwhile article. See useful Information on Shrug Ransomware.

    ReplyDelete
  4. thanks for sharing this Informative content. Well explained. Got to learn new things from your Blog on. SQL server dba Online Training

    ReplyDelete
  5. Are you tired of seeking loans and Mortgages,have you been turned down constantly By your banks and other financial institutions,We offer any form of loan to individuals and corporate bodies at low interest rate.If you are interested in taking a loan,feel free to contact us today,we promise to offer you the best services ever.Just give us a try,because a trial will convince you.What are your Financial needs?Do you need a business loan?Do you need a personal loan?Do you want to buy a car?Do you want to refinance?Do you need a mortgage loan?Do you need a huge capital to start off your business proposal or expansion? Have you lost hope and you think there is no way out, and your financial burdens still persists? Contact us (gaincreditloan1@gmail.com)

    Your Name:...............
    Your Country:...............
    Your Occupation:...............
    Loan Amount Needed:...............
    Loan Duration...............
    Monthly Income:...............
    Your Telephone Number:.....................
    Business Plan/Use Of Your Loan:...............
    Contact Us At : gaincreditloan1@gmail.com

    ReplyDelete
  6. Microsoft SQL Server 2019 Standard provides provides additional capability and improvements database features. like SQL Server database engine, SQL Server Analysis Services, SQL Server Machine Learning Services, SQL Server on Linux, and SQL Server Master Data Services. Microsoft SQL Server Standard can build rich content management applications

    ReplyDelete


  7. Nice Blog, Best Best microsoft office deals for Mac Home and Business edition is a powerful suite which fulfils the productivity applications, written for Mac OS X.

    ReplyDelete
  8. Wow finally I got good article about android development process from beginning. Thanks for sharing
    SQL Server DBA Training in Bangalore

    ReplyDelete