{"data":{"site":{"siteMetadata":{"title":"Slect"}},"allMarkdownRemark":{"edges":[{"node":{"excerpt":"\r\nBy the end of this post, you should know how to read and write from the following Spark's core data sources:
 1. Parquet.
 2.JDBC/ODBC.
 3. JSON
 4. CSV.
 5. Text\r\n\r\n","fields":{"slug":"/writing-data/"},"frontmatter":{"date":"October 15, 2019","title":".NET For Apache Spark - Writing Data."}}},{"node":{"excerpt":"\r\nBy the end of this post you should know how to do the following:
 1. Define the different cluster management systems Spark can run on.
 2. List the major components of Apache Hadoop YARN
 2. Explain the relationship between all major components of YARN
 3. Explain the functionalities of the Driver an Executors in Apache Spark.
 4. Explain how Spark uses cluster resource to process Big-Data\r\n\r\n","fields":{"slug":"/spark-architecture-part-1/"},"frontmatter":{"date":"September 24, 2019","title":"Apache Spark - Architecture Part 1: Driver and Executors"}}},{"node":{"excerpt":"\r\nBy the end of this post you should know how to do the following:
 1. List the different complex types available in Apache Spark
 2. How to query a DataFrame that has complex datatype
 \r\n\r\n","fields":{"slug":"/query-fundamentals-part-4/"},"frontmatter":{"date":"September 01, 2019","title":".Net for Apache Spark - DataFrame, Part 4: Complex Types"}}},{"node":{"excerpt":"\r\nThis post introduces you to the fundamentals of working with DataFrames.
By the end of this post you should know how to do the following with a Dataframe:
 1. Select specific columns.
 2. Filter rows.
 3. Add a Column.
 4. Remove a Column.
 5. Rename a Column.
 6. Change the Datatype of a Column.\r\n\r\n","fields":{"slug":"/query-fundamentals-part-3/"},"frontmatter":{"date":"August 25, 2019","title":".Net for Apache Spark - DataFrame, Part 3: Basic Transformations"}}},{"node":{"excerpt":"\r\nIn Apache Spark, business logic is expressed using Transformations.
\r\nThis blog post is a theoretical introduction to DataFrame Transformation\r\n\r\n","fields":{"slug":"/query-fundamentas-part-2/"},"frontmatter":{"date":"August 18, 2019","title":".Net for Apache Spark - DataFrame, Part 2: Transformations"}}},{"node":{"excerpt":"\r\nBy the end you should know how to do the following:
 1. Describe what a DataFrame is in Apache Spark.
 2. Explain what row and columns are.\r\n\r\n","fields":{"slug":"/query-fundamentals-part-1/"},"frontmatter":{"date":"August 11, 2019","title":".Net for Apache Spark - DataFrame, Part 1: Fundamentals"}}},{"node":{"excerpt":"\r\nBy the end of this post, you should know how to do the following:
 1. How to connect to SQL databases using JDBC .
 2. Specify a query that will be used to read data into Apache Spark
 3. Set a custom schema to use for reading data from a JDBC data source
\r\n\r\n","fields":{"slug":"/loading-data-part-3-rdbms/"},"frontmatter":{"date":"August 04, 2019","title":".Net for Apache Spark - Loading Data, Part 3 : RDBMS"}}},{"node":{"excerpt":"\r\nBy the end of this post, you should know how to do the following:
 1. Read JSON files.
 2. Configure options for the JSON format.
 3. Specify a complex DDL-formatted schema for a JSON file.
\r\n 4. Handle file with corrupted records\r\n\r\n","fields":{"slug":"/loading-data-part-2-json/"},"frontmatter":{"date":"July 30, 2019","title":".Net for Apache Spark - Loading Data, Part 2 : JSON files"}}},{"node":{"excerpt":"\r\nBy the end of this post, you should know how to do the following:
 1. Read csv file using format() and load().
 2. Configure options for the csv format.
 3. Specify a DDL-formatted schema for a csv file.\r\n\r\n","fields":{"slug":"/loading-data-part-1-csv/"},"frontmatter":{"date":"July 25, 2019","title":".Net for Apache Spark - Loading Data, Part 1 : CSV files"}}},{"node":{"excerpt":"\r\nApache Spark is an open source analytics engine for big data and machine learning.
\r\nApplication for Spark can be written in Scala, Python, R, SQL and now also for the\r\n.NET Core Framework using C# and F#.\r\n
\r\nIn this video, I will show you how to install Apache Spark on Windows and write an application using .NET Core and C# to test the installation\r\n\r\n","fields":{"slug":"/apache-spark-quick-start/"},"frontmatter":{"date":"June 01, 2019","title":".NET for Apache Spark - Getting Started"}}},{"node":{"excerpt":"\r\nBy the end of this video, you will be able to write queries to perform the following types of full-text searches:
1. Simple Term: find specific words 05:17
2. Prefix Term: query for words or phrases starting with a specific text 24:33
3. Generation Term: search for multiple forms of a specific word17:55
4. Synonymous Term: find words with similar meaning 28:58
\r\n\r\n ","fields":{"slug":"/full-text-search-middle/"},"frontmatter":{"date":"March 06, 2019","title":"SQL Server - Full text search with the Contains predicate"}}},{"node":{"excerpt":"\r\nSearching is the most fundamental thing we do with a database, most of the time we know the shape of what we are looking for, i.e an ID or specific text, but what if you can't state exactly what you are looking for?\r\n\r\n","fields":{"slug":"/full-text-search-intro/"},"frontmatter":{"date":"February 01, 2019","title":"SQL Server - Introduction to full-text search"}}}]}},"pageContext":{"isCreatedByStatefulCreatePages":true}}