tag:blogger.com,1999:blog-74688574629998450942024-03-18T08:33:06.884+05:30Microsoft SQL Server Knowledge BankThe Web Contains the Article related to Microsoft SQL ServerMR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.comBlogger455125tag:blogger.com,1999:blog-7468857462999845094.post-6411977552488524642020-06-03T16:20:00.000+05:302020-06-03T16:20:46.927+05:30New Blog Post Related to Azure ADF<div dir="ltr" style="text-align: left;" trbidi="on">
Hi Team,<br />
<br />
We have a new Blog Post for Azure Solution.<br />
<br />
<h2 style="text-align: left;">
<b><span style="color: blue;">SQL Knowledge Bank - Azure</span></b></h2>
<a href="https://mysqlknowledgebank.blogspot.com/">https://mysqlknowledgebank.blogspot.com</a><br />
<br />
Hope you can find lot over there.<br />
<br />
<br />
Regards,<br />
Joydeep Das</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com41tag:blogger.com,1999:blog-7468857462999845094.post-39311307664604337612019-03-23T10:26:00.002+05:302019-03-23T10:27:32.710+05:30SQL Server Performance Quick Fixer<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228851">Introduction</a></h1>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Performance
is a big issue for Microsoft SQL server. Every developer faces the Problem of
Query performance and they are try to tune the query accordingly.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Performance
of Microsoft SQL Server can be hampering for N number of reason including
hardware and networking. We have to find the proper reason behind it. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">It need time
to find the proper reason and provide an actual solution for that.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">It just likes
our real world situation. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Suppose we
have fever and we go to Doctor for treatment. Doctor knows very well that fever
is not a disease, it just a symptoms of disease.<span style="mso-spacerun: yes;"> </span>To find the root cause of the disease,
Doctors provide us some Tests but before that he needs to provide some quick
solution for Fever by providing tablets. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Here in this
article, we are trying to prove some quick fix of the Performance related
problem without finding the actual cause. We can also tell them the common
mistake that we have done with Microsoft SQL Server.<o:p></o:p></span></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228852">Quick Solution for Performance</a></h1>
<div class="MsoNormal">
<br /></div>
<h2>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228853">Auto Create Statistics in SQL Server</a></h2>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Most of the
Microsoft SQL Server Database does not have Auto Create Statistic On.<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">The
statistical histograms are used by the Query Optimizer to choose the optimal
query execution plan. If a query predicate contains a column with statistics,
the Query Optimizer does not have to estimate the number of rows affected by
that query, thus the Query Optimizer has enough information to create the
execution plan. <o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">The SQL
Server creates statistics in different ways:<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><br /></span></div>
<ul style="margin-top: 0in;" type="disc">
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l3 level1 lfo1; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">The statistics are automatically
created for each new index.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l3 level1 lfo1; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">If the database
setting AUTO_CREATE_STATISTICS is on, then the SQL Server will
automatically create statistics for non-indexed columns that are used in
your queries.<o:p></o:p></span></li>
</ul>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">When we set
the AUTO_CREATE_STATISTICS option on, the Query Optimizer creates statistics on
individual columns used in a predicate, if these statistics are not already
available. These statistics are necessary to generate the query plan. They are
created on columns that do not have a histogram in an existing statistics
object. <o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">These
statistics are used by the Query Optimizer to determine the optimal Query
Execution Plan. <o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">We can
enable the automatic statistics creation by running this SQL statement:<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">ALTER</span><span style="font-family: "consolas";"> <span style="color: blue;">DATABASE</span> <span style="color: teal;">TestDB</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">SET</span><span style="font-family: "consolas";"> <span style="color: blue;">AUTO_CREATE_STATISTICS</span> <span style="color: blue;">ON<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<h2>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228854">Auto Update Statistics in SQL Server</a></h2>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">Statistics
are checked before query compilation or before executing a cached query plan.
Statistics are considered out-of-date when:<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><br /></span></div>
<ul style="margin-top: 0in;" type="disc">
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l2 level1 lfo2; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">There was a data change on an
empty table.<o:p></o:p></span></li>
</ul>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 0in;">
<br /></div>
<ul style="margin-top: 0in;" type="disc">
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l2 level1 lfo2; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">The number of rows in the table
was 500 or less at the time of statistics creation and the column
modification counter of the leading column of the statistics object has
changed by more than 500 since then.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l2 level1 lfo2; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">The table had more than 500 rows
when the statistics were gathered, and the column modification counter of
the leading column of the statistics object has changed by more than 500 +
20% of the number of rows in the table when the statistics were gathered.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-list: l2 level1 lfo2; tab-stops: list .5in;"><span style="font-family: "arial" , "sans-serif";">A table in TempDB with less than
6 rows had at least 6 row modifications.<o:p></o:p></span></li>
</ul>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">We can turn
on automatic statistics update by running this SQL statement:<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">ALTER</span><span style="font-family: "consolas";"> <span style="color: blue;">DATABASE</span> <span style="color: teal;">TestDB</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">SET</span><span style="font-family: "consolas";"> <span style="color: blue;">AUTO_UPDATE_STATISTICS</span> <span style="color: blue;">ON<o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">The
outdated statistics can cause a lot of </span><span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; text-decoration: none;">performance issues</span><span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"> therefore it is recommended
to enable it. The default option is ON. The usual symptoms of non-updated
statistics are suboptimal query plans and degraded performance. You can also
set the database to update statistics asynchronously:<o:p></o:p></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">ALTER</span><span style="font-family: "consolas";"> <span style="color: blue;">DATABASE</span> <span style="color: teal;">TestDB</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">SET</span><span style="font-family: "consolas";"> <span style="color: blue;">AUTO_UPDATE_STATISTICS_ASYNC</span> <span style="color: blue;">ON<o:p></o:p></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="color: blue;"><br /></span></span></div>
<div style="background: white;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;"><span style="mso-spacerun: yes;"> </span>If we enable this option then the Query
Optimizer will run the query first and update the outdated statistics
afterwards. When you set this option to OFF, the Query Optimizer will update
the outdated statistics before compiling the query. This option can be useful
in OLTP environments while it can have negative effects in data warehouses. <o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228855">Database AutoGrowth</a></h1>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">We use SQL
Server database </span><span style="mso-bidi-font-weight: bold;">Autogrowth</span><span style="font-family: "arial" , "sans-serif";"> setting to automate database
file growth. This property is very helpful if you use it carefully with proper
planning. But there can be a negative performance impact on your database if we
don’t give attention on it and leave it with default values. If we have too
many autogrow events in our database, it can degrade the overall performance of
the database. Below are the points that we can consider to reduce the autogrow
events in a database to improve performance.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Auto
growth events are expensive operations that slow down
the performance of our database because whenever
an auto-growth event is performed, SQL Serverholds up database
processing. This equates to slower response time for those SQLcommands
that are being processing against the database that is growing.<o:p></o:p></span></div>
<h2>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228856">Best Practice</a></h2>
<div style="background: white; margin-bottom: 12.0pt; margin-left: 0in; margin-right: 0in; margin-top: 0in; vertical-align: baseline;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">We can improve the database performance and can manage the disk
space utilization by focusing on below points. This will also reduce the number
of Autogrowth events to be occurred in the database files.<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Analyze your database growth pattern
and set optimum size of database with the analyzed growth settings. Our
database should have enough size that it never grows and if it grows make
sure that </span>I<span style="font-family: "arial" , "sans-serif";">nstant
file initialization should be enabled on that SQL Server Instance. This
will drastically reduce the Autogrowth events in day to day life.<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; vertical-align: baseline;">
<br /></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Proactively monitor database file
sizes and their autogrowth events. This helps us to analyze the growth pattern
of database files and avoid fragmentation.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">3.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Consider defragmenting your database
file system if we have lot of auto-growth events occurred on your databases.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">4.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Never leave your database autogrowth
with default values, change it as per growth pattern that you analyzed over
time.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">5.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Avoid using autogrowth in percentage;
rather give a specific amount of size in MB/GB. Microsoft suggests to set
your autogrow setting to about one-eighth the size of the file and
test it before deploying on PROD.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">6.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Turn on the <MAXSIZE> setting
for each file to prevent any one file from growing to a point where it uses up
all available disk space.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 22.5pt; margin-right: 0in; margin-top: 0in; mso-list: l4 level1 lfo3; tab-stops: list .5in; text-indent: -.25in; vertical-align: baseline;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">7.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">The growth increment of our
transaction log must be large enough to stay ahead of the needs of your
transaction units. Even with autogrow turned on, you can receive a
message that the transaction log is full, if it cannot grow fast enough to
satisfy the needs of your query.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228857">Identifying the Missing Index</a></h1>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">When we think
about the performance of the query, the very first time we have to think about
index. Finding the proper column of a table for indexing is a challenge.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">We can
identify missing indexes in your SQL queries by 3 ways primarily —<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo4; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Running
the Database Engine Tuning Advisor<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo4; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Executing
Missing Index Dynamic Management Views<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo4; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">SQL
Server Engine prompts missing indexes when you generate Execution Plans in SSMS<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">SELECT</span><span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: blue;">DISTINCT</span> <span style="color: magenta;">CONVERT</span><span style="color: grey;">(</span><span style="color: blue;">DECIMAL</span><span style="color: grey;">(</span>18<span style="color: grey;">,</span> 2<span style="color: grey;">)</span> <span style="color: grey;">,</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">user_seeks</span> <span style="color: grey;">*</span> <span style="color: teal;">avg_total_user_cost</span> <span style="color: grey;">*</span><span style="color: blue;"> </span><span style="color: grey;">(</span> <span style="color: teal;">avg_user_impact</span> <span style="color: grey;">*</span>
0.01 <span style="color: grey;">))</span> <span style="color: blue;">AS</span><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">[index_advantage]</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">last_user_seek</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">[statement]</span> <span style="color: blue;">AS</span> <span style="color: teal;">[Database.Schema.Table]</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">equality_columns</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">inequality_columns</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">included_columns</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">unique_compiles</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">user_seeks</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">avg_total_user_cost</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">avg_user_impact</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: magenta;">OBJECT_NAME</span><span style="color: grey;">(</span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">[object_id]</span><span style="color: grey;">)</span> <span style="color: blue;">AS</span> <span style="color: teal;">[Table Name]</span> <span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">p</span><span style="color: grey;">.</span><span style="color: blue;">rows</span> <span style="color: blue;">AS</span> <span style="color: teal;">[Table Rows]</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">FROM</span><span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">dm_db_missing_index_group_stats</span>
<span style="color: blue;">AS</span> <span style="color: teal;">migs</span> <span style="color: blue;">WITH </span><span style="color: grey;">(</span> <span style="color: blue;">NOLOCK</span> <span style="color: grey;">)</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">INNER</span> <span style="color: grey;">JOIN</span> <span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">dm_db_missing_index_groups</span> <span style="color: blue;">AS</span>
<span style="color: teal;">mig</span> <span style="color: blue;">WITH </span><span style="color: grey;">(</span> <span style="color: blue;">NOLOCK</span> <span style="color: grey;">)</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: blue;">ON</span> <span style="color: teal;">migs</span><span style="color: grey;">.</span><span style="color: teal;">group_handle</span> <span style="color: grey;">=</span> <span style="color: teal;">mig</span><span style="color: grey;">.</span><span style="color: teal;">index_group_handle</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">INNER</span> <span style="color: grey;">JOIN</span> <span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">dm_db_missing_index_details</span> <span style="color: blue;">AS</span>
<span style="color: teal;">mid</span> <span style="color: blue;">WITH </span><span style="color: grey;">(</span> <span style="color: blue;">NOLOCK</span> <span style="color: grey;">)</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: blue;">ON</span> <span style="color: teal;">mig</span><span style="color: grey;">.</span><span style="color: teal;">index_handle</span> <span style="color: grey;">=</span> <span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">index_handle</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">INNER</span> <span style="color: grey;">JOIN</span> <span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">partitions</span> <span style="color: blue;">AS</span> <span style="color: teal;">p</span> <span style="color: blue;">WITH </span><span style="color: grey;">(</span> <span style="color: blue;">NOLOCK</span> <span style="color: grey;">)</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: blue;">ON</span> <span style="color: teal;">p</span><span style="color: grey;">.</span><span style="color: teal;">[object_id]</span> <span style="color: grey;">=</span> <span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">[object_id]</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">WHERE</span><span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: teal;">mid</span><span style="color: grey;">.</span><span style="color: teal;">database_id</span> <span style="color: grey;">=</span> <span style="color: magenta;">DB_ID</span><span style="color: grey;">()</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">ORDER</span><span style="font-family: "consolas";"> <span style="color: blue;">BY</span> <span style="color: teal;">index_advantage</span> <span style="color: blue;">DESC</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">OPTION </span><span style="color: grey; font-family: "consolas";">(</span><span style="font-family: "consolas";"> <span style="color: blue;">RECOMPILE</span> <span style="color: grey;">);<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228858">Identify Unused Index</a></h1>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">We are
creating Index to improve the performance of SQL query. What happens if the
Index is not used by SQL Server or we are creating index in those column that
are not needed to create index. Moreover by creating index we decrease the
performance of Insert/Update. Over indexing is also bad.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">SQL Server
provides a significant amount of index information via Dynamic Management Views
(DMVs). The dm_db_index_usage_stats DMV displays essential information about
index usage, and it can be a useful tool in identifying unused SQL Server
indexes. When an index is used for the first time, a new row gets created in
the dm_db_index_usage_stats DMV and subsequently updated every time an index is
used. However, as with every DMV, the data present in <b>dm_db_index_usage_stats</b> contain
only the data since the last SQL Server service restart (SQL Server service
restart resets the data in the DMV). Therefore, it is critical that there is a
sufficient time since the last SQL Server restart that allows correctly
determining which indexes are good candidates to be dropped<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">SELECT</span><span style="font-family: "consolas";"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">objects</span><span style="color: grey;">.</span><span style="color: teal;">name</span> <span style="color: blue;">AS</span> <span style="color: teal;">Table_name</span><span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">indexes</span><span style="color: grey;">.</span><span style="color: teal;">name</span> <span style="color: blue;">AS</span> <span style="color: teal;">Index_name</span><span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_seeks</span><span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_scans</span><span style="color: grey;">,</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_updates</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">FROM</span><span style="font-family: "consolas";"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">dm_db_index_usage_stats</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">INNER</span>
<span style="color: grey;">JOIN</span> <span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">objects</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-tab-count: 1;"> </span><span style="color: blue;">ON</span>
<span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: magenta;">OBJECT_ID</span> <span style="color: grey;">=</span> <span style="color: green;">objects</span><span style="color: grey;">.</span><span style="color: magenta;">OBJECT_ID</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">INNER</span>
<span style="color: grey;">JOIN</span> <span style="color: green;">sys</span><span style="color: grey;">.</span><span style="color: green;">indexes</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-tab-count: 1;"> </span><span style="color: blue;">ON</span>
<span style="color: green;">indexes</span><span style="color: grey;">.</span><span style="color: teal;">index_id</span> <span style="color: grey;">=</span> <span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">index_id</span> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-tab-count: 1;"> </span><span style="mso-spacerun: yes;"> </span><span style="color: grey;">AND</span> <span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: magenta;">OBJECT_ID</span> <span style="color: grey;">=</span> <span style="color: green;">indexes</span><span style="color: grey;">.</span><span style="color: magenta;">OBJECT_ID</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">WHERE</span><span style="font-family: "consolas";"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">indexes</span><span style="color: grey;">.</span><span style="color: teal;">is_primary_key</span> <span style="color: grey;">=</span> 0 <span style="color: green;">-- This condition
excludes primary key constarint</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">AND</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">indexes</span><span style="color: grey;">.</span> <span style="color: teal;">is_unique</span> <span style="color: grey;">=</span> 0 <span style="color: green;">-- This condition
excludes unique key constarint</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">AND</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span> <span style="color: teal;">user_lookups</span> <span style="color: grey;">=</span> 0<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">AND</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_seeks</span> <span style="color: grey;">=</span> 0<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: grey;">AND</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_scans</span> <span style="color: grey;">=</span> 0<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas";">ORDER</span><span style="font-family: "consolas";"> <span style="color: blue;">BY</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas";"><span style="mso-spacerun: yes;"> </span><span style="color: green;">dm_db_index_usage_stats</span><span style="color: grey;">.</span><span style="color: teal;">user_updates</span> <span style="color: blue;">DESC<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228859">Using Proper Data Type</a> </h1>
<div style="background: white; line-height: 18.85pt; margin-bottom: 12.0pt; margin-left: 0in; margin-right: 0in; margin-top: 0in;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">Did you know that poor data type choices can have a significant
impact on our database design and performance. Developers and DBAs can
improve database performance by understanding the data types supported by SQL
Server and the implications of choosing different ones. A best practice is to
“right size” data types by asking business questions and determining the data
types that are best suited to the organization’s and application’s needs.<o:p></o:p></span></div>
<div style="-webkit-font-smoothing: antialiased; -webkit-text-stroke-width: 0px; background: white; box-sizing: border-box; font-variant-caps: normal; font-variant-ligatures: normal; line-height: 18.85pt; margin-bottom: 12.0pt; margin-left: 0in; margin-right: 0in; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt;">Right-sizing can result in tremendous storage savings, which can
lead to faster database performance. Other considerations to keep in mind are
SQL Server’s 8KB data page size limit and the conditions that can lead to page
splitting. Also watch out for implicit conversions, an unfortunate by-product
of mismatching data types. Taking steps to avoid mismatches and page splitting
can greatly enhance performance.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc4228860">Clean T-SQL</a></h1>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">When writing
T-SQL code we have to make it simple. If we make the complex SQL (Suppose
joining so many table), the SQL optimizer don’t get the proper time to make
multiple cost execution plan and choosing the most effective one. So we must
keep our SQL as simple as possible. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">We have to
check the EXECUTION PLAN and try to understand the root cause of the poor
performance of the query.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">There are several
things that we have to take care when writing SQL Statement. Some of them are mentioned
bellow.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
use SELECT * statement. Use the proper column in SELECT statement<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Try
to make JOIN within the column that have INDEX<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">3.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
use any UDF within the SQL Statement<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">4.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
use any function within WHERE clause<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">5.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
use any Co-Related Sub query<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">6.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
try to join a lot of table together <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">7.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Don’t
use IN clause, use EXISTS<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">8.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">In
Stored procedure try to use SET NOCOUNT ON<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">9.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">We
are not preferring any APPLY Clause in SQL statement. Try to convert it in JOIN<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">10.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Avoid Cursor and always try to make
SET Based SQL <br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">11.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Try to make Logic as Simple as
Possible<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">12.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">If needed use Temp Table<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">13.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Try to avoid DISTINCT, ORDER BY Clause<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo5; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "arial" , "sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">14.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "arial" , "sans-serif";">Try to use Proper data type in Table
and try to avoid the Data conversion in SQL statement<o:p></o:p></span></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com17tag:blogger.com,1999:blog-7468857462999845094.post-34948918488454370482018-10-06T12:11:00.000+05:302018-10-06T12:11:12.690+05:30Debugging and Remote Submission of Spark Job in HDInsight Spark Cluster<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<h1>
<a href="https://www.blogger.com/u/2/null" name="_Toc526590420">Introduction</a></h1>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">In this
article we are trying to create a Scala program in IntelliJ and debug it
locally and then debug the same program within HDInsight spark cluster and
finally submit the Spark Job within HDInsight spark cluster remotely.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Steps Involved:<o:p></o:p></span></div>
<ol start="1" type="1">
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">From IntelleJ create a
application that can read data from Blob storage and write count output in
blob storage<o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">Run the file in HDInsight
(IntelleJ HDInsight submission)<o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">Create JAR file<o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">Upload JAR file into BLOB storage<o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">Install cURL utility for command
line sumbission<o:p></o:p></span></li>
<li class="MsoNormal" style="background: white; mso-list: l0 level1 lfo1; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in;"><span style="font-family: "Arial","sans-serif";">Submit the SPARK job with cURL
command line.<o:p></o:p></span></li>
</ol>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><u><span style="color: #c00000; font-family: "Arial","sans-serif";">Our HDInsight Spark
Cluster should be BLOB as Primary Storage. This Process NOT work if the
HDInsight Spark cluster Primary storage is Data Lake Storage (ADLS) <o:p></o:p></span></u></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><u><span style="color: #c00000; font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><u><span style="color: #c00000; font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: Arial, sans-serif;"><span style="color: blue;">Download Link:</span></span></b></div>
<h1>
<br /></h1>
<div class="MsoNormal">
<a href="https://drive.google.com/file/d/1dMKsgH1dpYsMtgZu2tzhEinxtxGS3d2d/view?usp=sharing">https://drive.google.com/file/d/1dMKsgH1dpYsMtgZu2tzhEinxtxGS3d2d/view?usp=sharing</a></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-50343350131076171502018-06-24T09:37:00.000+05:302018-06-24T09:37:10.265+05:30Spark working with Unstructured data<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">In my
previous article with Spark, we worked with structure and semi structure data
source. Here in this article we are trying to work with unstructured data
source. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Case Study<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We have a
note book and we want to find number of work count in it. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJtZpgl1QIgHo1KVBZLnEc0Gqz8fQn5oi5e_TQHTAoPmhnETFtKyLD1_VZrmpmjyXCv-4tLT62_Hpj9JEMfmHrPlhP7J0aR13JrVGI2uH6KGYXD379qXCQqDUDdD5YrfNJ5Yi7Gjy6_yc/s1600/fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="148" data-original-width="507" height="116" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJtZpgl1QIgHo1KVBZLnEc0Gqz8fQn5oi5e_TQHTAoPmhnETFtKyLD1_VZrmpmjyXCv-4tLT62_Hpj9JEMfmHrPlhP7J0aR13JrVGI2uH6KGYXD379qXCQqDUDdD5YrfNJ5Yi7Gjy6_yc/s400/fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala Code<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> dfsFilename = </span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"D:/spark/bin/examples/src/main/resources/notebook.txt"</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> text =
sc.textFile(dfsFilename)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> counts =
text.flatMap(line => line.split(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">" "</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)).map(word => (word,</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">1</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)).reduceByKey(_+_) </span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">counts</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">.collect.foreach(println)<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Output<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaQQNr5OV1UN_qkH1glg52KbbP-Qn3YQcDskRbEdLMBrr_TsH8Oo_N-luusCF_AVhlwYOZ8hHHStC1eKq06GOAolrVfufLdG7egqfpcOp5lU66eC3c8gpRq7j0dZqfFi-MMgTQ_XdHKw0/s1600/fig-2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="450" data-original-width="643" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaQQNr5OV1UN_qkH1glg52KbbP-Qn3YQcDskRbEdLMBrr_TsH8Oo_N-luusCF_AVhlwYOZ8hHHStC1eKq06GOAolrVfufLdG7egqfpcOp5lU66eC3c8gpRq7j0dZqfFi-MMgTQ_XdHKw0/s400/fig-2.JPG" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com12tag:blogger.com,1999:blog-7468857462999845094.post-67360653016705438152018-06-20T20:40:00.001+05:302018-06-20T20:40:28.334+05:30Spark to Connect with Azure SQL DB and read Table<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">In this
article we are trying to connect spark with Azure SQL DB and just simply read a
table.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala Code<o:p></o:p></span></b></div>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="background: white; border-collapse: collapse; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0;">
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 1;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">import</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">com</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">microsoft</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">azure</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">sqldb</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">spark</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">bulkcopy</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">BulkCopyMetadata</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 2;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">import</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">com</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">microsoft</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">azure</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">sqldb</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">spark</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">config</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Config</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 3;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">import</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">com</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">microsoft</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">azure</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">sqldb</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">spark</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">connect</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">_<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 4;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 5;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">url</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"[Enter
your url here]"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 6;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">databaseName</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"[Enter
your database name here]"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 7;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">dbTable</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"[Enter
your database table here]"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 8;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 9;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">user</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"[Enter
your username here]"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 10;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">password</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"[Enter
your password here]"<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<br /></div>
</td>
</tr>
<tr style="mso-yfti-irow: 11;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 12;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #6a737d; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">// READ
FROM CONFIG</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 13;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">readConfig</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Config</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">(</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Map</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">(<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 14;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"url"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> url,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 15;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"databaseName"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> databaseName,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 16;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"user"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> user,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 17;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"password"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> password,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 18;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"connectTimeout"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"5"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 19;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"queryTimeout"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"5"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">,<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 20;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"dbTable"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><span style="mso-spacerun: yes;"> </span></span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">-></span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> dbTable<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 21;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">))<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 22;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 23;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">df</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> sqlContext.read.sqlDB(readConfig)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 24;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">println(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"Total rows: "</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">
</span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">+</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> df.count)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 25;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">df.show()<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 26;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 27;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #6a737d; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">//
TRADITIONAL SYNTAX</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 28;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">import</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">java</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">util</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">.</span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Properties<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 29;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 30;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">properties</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">new</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #6f42c1; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">Properties</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">()<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 31;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">properties.put(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"databaseName"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">, databaseName)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 32;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">properties.put(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"user"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">, user)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 33;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">properties.put(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"password"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">, password)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 34;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">properties.put(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"connectTimeout"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">, </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"5"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 35;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">properties.put(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"queryTimeout"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">, </span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"5"</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 36;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top"></td>
</tr>
<tr style="mso-yfti-irow: 37;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">val</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">df</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> </span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">=</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> sqlContext.read.sqlDB(url, dbTable, properties)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 38;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">println(</span><span style="color: #032f62; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">"Total rows: "</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">
</span><span style="color: #d73a49; font-family: Consolas; font-size: 9.0pt; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: "Times New Roman";">+</span><span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"> df.count)<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 39; mso-yfti-lastrow: yes;">
<td nowrap="" style="padding: 0in 7.5pt 0in 7.5pt; width: 37.5pt;" valign="top" width="50"></td>
<td style="padding: 0in 7.5pt 0in 7.5pt;" valign="top">
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";">df.show()<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<br /></div>
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p><br /></o:p></span></div>
<div class="MsoNormal" style="line-height: 15.0pt; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #24292e; font-family: Consolas; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman";"><o:p><br /></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com6tag:blogger.com,1999:blog-7468857462999845094.post-70023349952096518092018-06-18T20:51:00.002+05:302018-06-18T20:51:44.292+05:30SSIS Folder Traversing in SPARK SQL<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Here in this
article, we are trying to demonstrate Folder Traversing of SSIS ForEach loop
container for searching a specified file.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be interesting<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scenario<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We have a
folder named “Sample”. Under this folder, we have three other folder named “Sample-1”,
“Sample-2” and “Sample-3”. For each folder there is a flat file named “Student-1.txt”,”Student-2.txt”
and “Student-3.txt”. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We need to
read the entire file from different folder location<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">The folder
and file structure is displayed by DOS TREE command<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1K0b96mflxWik797obLayNFZ-8639fJPLnppXVk5M3ThtfqkydLRUQFh17Ik8dv1zylfMSnE_lqnQlnExvsntr25q1iTU9_hXFQqYApbSmWbewvKN8zcafmDHm_PaLYPNkmoY4OThBbQ/s1600/fig-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="131" data-original-width="264" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1K0b96mflxWik797obLayNFZ-8639fJPLnppXVk5M3ThtfqkydLRUQFh17Ik8dv1zylfMSnE_lqnQlnExvsntr25q1iTU9_hXFQqYApbSmWbewvKN8zcafmDHm_PaLYPNkmoY4OThBbQ/s400/fig-1.JPG" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala Code<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//---------------------------------------</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// <u>Scala</u> for SPARK to Read
Flat File form Different Folder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Implementation Folder Traversing
of SSIS in Spark</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Creation Date: 06/18/2018</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//-----------------------------------------</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.SparkSession</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.catalyst.encoders.ExpressionEncoder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.Encoder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>spark</u>.implicits._</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">case</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">class</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><b><span style="color: #a22e00; font-family: Consolas; font-size: 10.0pt;">Student</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">roll</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: Long, </span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">name</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: </span><i><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">String</span></i><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> employeeDF =
spark.sparkContext.textFile(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"d:/spark/bin/examples/src/main/resources/sample/*/student-*.txt"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).map(_.split(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">","</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)).map(attributes=>Student(attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">0</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim.toInt,
attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">1</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim)).toDF()</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">employeeDF</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.createOrReplaceTempView(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"student"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> youngstersDF =
spark.sql(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"SELECT
roll, name FROM student"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">youngstersDF</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">.show</span><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Output<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1ST-mGSD-C8agfPxoA8RzVPK9APDMIqM_U6ncjo7LEOUYvIzJfKkYNTZfhfW2quNM7xjfgDo1vGzQ-U9In-oTpo6iA5549D0UGORoWj-2UxwSlyWaKxflOTiFi5rmqla4FeQqrjArv98/s1600/fig-2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="252" data-original-width="640" height="157" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1ST-mGSD-C8agfPxoA8RzVPK9APDMIqM_U6ncjo7LEOUYvIzJfKkYNTZfhfW2quNM7xjfgDo1vGzQ-U9In-oTpo6iA5549D0UGORoWj-2UxwSlyWaKxflOTiFi5rmqla4FeQqrjArv98/s400/fig-2.JPG" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
By: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-77683609198667590472018-06-17T15:04:00.001+05:302018-06-17T15:04:38.442+05:30SSIS Conditional Split with SPARK SQL<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Here in this
article we are trying to make SSIS conditional Split Transform by using SPARK
SQL. It is called Split Data Frame using Filter Transform.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">What We Want to Do<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We have a
Flat File Named Student Marks Details mentioned bellow.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">101,Joydeep Das,Math,10<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">102,Deepasree Das,Math,89<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">103,Shipra Roy
Chowdhury,Math,100<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">104,Rajesh Roy,Math,45<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">105,Sunita Tendon,Math,98<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">106,Amit Basu,Math,20<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">107,Debalina
Bhattacharya,Math,99<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">108,Pritam Das,Math,40<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">109,Partha Deb Das,Math,89<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="color: #002060;">110,Onkita Gupta,Math,100<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br />
The file contains Student Roll, Student Name, Subject and Marks. Based on Marks
we need to display records<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0;">
<td style="border: solid black 1.0pt; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 239.4pt;" valign="top" width="319">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial","sans-serif";">If [ Marks ] > 50<o:p></o:p></span></div>
</td>
<td style="border-left: none; border: solid black 1.0pt; mso-border-alt: solid black .5pt; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-themecolor: text1; mso-border-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 239.4pt;" valign="top" width="319">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial","sans-serif";">Display Records<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 1; mso-yfti-lastrow: yes;">
<td style="border-top: none; border: solid black 1.0pt; mso-border-alt: solid black .5pt; mso-border-themecolor: text1; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 239.4pt;" valign="top" width="319">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial","sans-serif";">If [ Marks ] < 50<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: solid black 1.0pt; border-top: none; mso-border-alt: solid black .5pt; mso-border-bottom-themecolor: text1; mso-border-left-alt: solid black .5pt; mso-border-left-themecolor: text1; mso-border-right-themecolor: text1; mso-border-themecolor: text1; mso-border-top-alt: solid black .5pt; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 239.4pt;" valign="top" width="319">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial","sans-serif";">Display Records<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala Code<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//---------------------------------------</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Scala for SPARK to Read Flat File</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Make Conditional Split Depends on
Makes (Marks>50)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Creation Date: 06/172018</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//-----------------------------------------</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.SparkSession</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.catalyst.encoders.ExpressionEncoder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.Encoder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> <u>spark</u>.implicits._</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">case</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">class</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><b><span style="color: #a22e00; font-family: Consolas; font-size: 10.0pt;">Student</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">roll</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: Long, </span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">name</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: </span><i><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">String</span></i><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">subject</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: </span><i><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">String</span></i><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">marks</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: Long)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//Read FLAT File</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> studentDF =
spark.sparkContext.textFile(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"d:/spark/bin/examples/src/main/resources/studentmarksdetails.txt"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).map(_.split(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">","</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)).map(attributes
=> Student(attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">0</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim.toInt, attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">1</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim, attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">2</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim, attributes(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">3</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).trim.toInt)).toDF()</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">studentDF</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.cache() </span><span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// recommended to
prevent repeating the calculation</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> condition = col(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"marks"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">) > </span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">50</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">// Condition</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> studentDF1 =
studentDF.filter(condition)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> studentDF2 =
studentDF.filter(not(condition))</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: Consolas; font-size: 10.0pt;">//Making View for FLAT file</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">studentDF1</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.createOrReplaceTempView(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"studentrecord1"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">studentDF2</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.createOrReplaceTempView(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"studentrecord2"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> youngstersDF1 =
spark.sql(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"SELECT
roll, name, subject, marks FROM studentrecord1"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">youngstersDF1</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.show</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> youngstersDF2 =
spark.sql(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"SELECT
roll, name, subject, marks FROM studentrecord2"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">youngstersDF2</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">.show</span><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Output<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipTEdMjtlyEZX-Li6aisqAAFy6rP7OCek_B59tUgIiutsfXOo1_RvZbV4R6UyDGJvCUB6G13wKdXemQ7-bH1KAFGsBVoHJa6uiGlS0AxX496kAL2iq5wcSvx0bqgUD3gGfVDRtTLV_vBo/s1600/FIG-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="480" data-original-width="644" height="297" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipTEdMjtlyEZX-Li6aisqAAFy6rP7OCek_B59tUgIiutsfXOo1_RvZbV4R6UyDGJvCUB6G13wKdXemQ7-bH1KAFGsBVoHJa6uiGlS0AxX496kAL2iq5wcSvx0bqgUD3gGfVDRtTLV_vBo/s400/FIG-1.JPG" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-50610395578005888042018-06-15T20:38:00.000+05:302018-06-15T20:38:41.067+05:30Download JSON file from Azure Storage and Read it by SSIS – Part 2<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="color: #222222; font-family: "arial" , sans-serif; font-size: 14.0pt;">Introduction<br />
<!--[if !supportLineBreakNewLine]--><br />
<!--[endif]--></span></b><span style="color: #222222; font-family: "arial" , sans-serif; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;">This article is
the continuation of my previous article named “Download JSON file from Azure
Storage and Read it by SSIS”.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;">You can find
this article:<br />
<a href="http://sqlknowledgebank.blogspot.in/2018/04/download-json-file-from-azure-storage.html">http://sqlknowledgebank.blogspot.in/2018/04/download-json-file-from-azure-storage.html</a><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;">In this article
we are going to read the JSON file and store it in Relational Database Table
object. For that we are using <b>Script Component</b>.
Hope it will be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;"><br /></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="color: #222222; font-family: "arial" , sans-serif; font-size: 14.0pt;">JSON File<br />
<!--[if !supportLineBreakNewLine]--><br />
<!--[endif]--><o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;">The sample of
the example JSON file<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , sans-serif;"><br /></span></div>
<div class="MsoNormal">
<!--[if gte vml 1]><v:shapetype
id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t"
path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter"/>
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0"/>
<v:f eqn="sum @0 1 0"/>
<v:f eqn="sum 0 0 @1"/>
<v:f eqn="prod @2 1 2"/>
<v:f eqn="prod @3 21600 pixelWidth"/>
<v:f eqn="prod @3 21600 pixelHeight"/>
<v:f eqn="sum @0 0 1"/>
<v:f eqn="prod @6 1 2"/>
<v:f eqn="prod @7 21600 pixelWidth"/>
<v:f eqn="sum @8 21600 0"/>
<v:f eqn="prod @7 21600 pixelHeight"/>
<v:f eqn="sum @10 21600 0"/>
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect"/>
<o:lock v:ext="edit" aspectratio="t"/>
</v:shapetype><v:shape id="Picture_x0020_1" o:spid="_x0000_i1029" type="#_x0000_t75"
style='width:468pt;height:69pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:///C:\Users\569442\AppData\Local\Temp\msohtmlclip1\01\clip_image001.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--><span style="font-family: "arial" , sans-serif;"><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="color: #222222; font-family: "arial" , sans-serif; font-size: 14.0pt;">Data Flow Task<br />
<!--[if !supportLineBreakNewLine]--><br />
<!--[endif]--><o:p></o:p></span></b></div>
<div class="MsoNormal">
<!--[if gte vml 1]><v:shape
id="Picture_x0020_2" o:spid="_x0000_i1028" type="#_x0000_t75" style='width:205.5pt;
height:169.5pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:///C:\Users\569442\AppData\Local\Temp\msohtmlclip1\01\clip_image003.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><img border="0" height="226" src="file:///C:/Users/569442/AppData/Local/Temp/msohtmlclip1/01/clip_image003.png" v:shapes="Picture_x0020_2" width="274" /><!--[endif]--><span style="font-family: "arial" , sans-serif;"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="color: #222222; font-family: "arial" , sans-serif; font-size: 14.0pt; line-height: 107%;">Script Component Settings<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b><u>Connection Manager
with JSON File<o:p></o:p></u></b></div>
<div class="MsoNormal">
<!--[if gte vml 1]><v:shape
id="Picture_x0020_3" o:spid="_x0000_i1027" type="#_x0000_t75" style='width:468pt;
height:197.25pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:///C:\Users\569442\AppData\Local\Temp\msohtmlclip1\01\clip_image004.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><img border="0" height="263" src="file:///C:/Users/569442/AppData/Local/Temp/msohtmlclip1/01/clip_image004.png" v:shapes="Picture_x0020_3" width="624" /><!--[endif]--><o:p></o:p></div>
<div class="MsoNormal">
<b><u>Input Output
Columns<o:p></o:p></u></b></div>
<div class="MsoNormal">
<!--[if gte vml 1]><v:shape
id="Picture_x0020_4" o:spid="_x0000_i1026" type="#_x0000_t75" style='width:468pt;
height:270pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:///C:\Users\569442\AppData\Local\Temp\msohtmlclip1\01\clip_image005.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><img border="0" height="360" src="file:///C:/Users/569442/AppData/Local/Temp/msohtmlclip1/01/clip_image005.png" v:shapes="Picture_x0020_4" width="624" /><!--[endif]--><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><u>Edit Script<o:p></o:p></u></b></div>
<div class="MsoNormal">
References needed <b>Syste.Web.Entry</b><o:p></o:p></div>
<div class="MsoNormal">
<!--[if gte vml 1]><v:shape
id="Picture_x0020_5" o:spid="_x0000_i1025" type="#_x0000_t75" style='width:346.5pt;
height:257.25pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:///C:\Users\569442\AppData\Local\Temp\msohtmlclip1\01\clip_image006.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><img border="0" height="343" src="file:///C:/Users/569442/AppData/Local/Temp/msohtmlclip1/01/clip_image006.png" v:shapes="Picture_x0020_5" width="462" /><!--[endif]--><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Name Spaces needed:<o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System.Data;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Pipeline.Wrapper;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Runtime.Wrapper;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System.Web.Script.Serialization;<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
C# Code:<o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">#region</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Help:
Introduction to the Script Component<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">/* The Script Component allows you to
perform virtually any operation that can be accomplished in</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> * a .Net application within the context of an
Integration Services data flow.</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> *</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> * Expand the other regions which have
"Help" prefixes for examples of specific ways to use</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> * Integration Services features within this
script component. */</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">#endregion</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">#region</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Namespaces<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System.Data;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Pipeline.Wrapper;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Runtime.Wrapper;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">using</span><span style="font-family: "consolas"; font-size: 9.5pt;"> System.Web.Script.Serialization;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">#endregion</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: grey; font-family: "consolas"; font-size: 9.5pt;">///</span><span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> </span><span style="color: grey; font-family: "consolas"; font-size: 9.5pt;"><summary></span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: grey; font-family: "consolas"; font-size: 9.5pt;">///</span><span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> This is the class to which to add your
code. Do not change the name,
attributes, or parent</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: grey; font-family: "consolas"; font-size: 9.5pt;">///</span><span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> of this class.</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: grey; font-family: "consolas"; font-size: 9.5pt;">///</span><span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> </span><span style="color: grey; font-family: "consolas"; font-size: 9.5pt;"></summary></span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">[Microsoft.SqlServer.Dts.Pipeline.<span style="color: #2b91af;">SSISScriptComponentEntryPointAttribute</span>]<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">public</span><span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">class</span> <span style="color: #2b91af;">ScriptMain</span> : <span style="color: #2b91af;">UserComponent</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">
</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: #2b91af;">IDTSConnectionManager100</span>
connMgr;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> </span><span style="color: grey;"><summary></span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> This method is called
once, before rows begin to be processed in the data flow.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> You can remove this
method if you don't need to do anything here.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> </span><span style="color: grey;"></summary></span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">override</span> <span style="color: blue;">void</span> PreExecute()<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">base</span>.PreExecute();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: green;">/*</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">
* Add your code here</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">
*/</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
connMgr = <span style="color: blue;">this</span>.Connections.MyFlatFileConnectionMgr;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> </span><span style="color: grey;"><summary></span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> This method is called
after all the rows have passed through this component.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> You can delete this
method if you don't need to do anything here.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: grey;">///</span><span style="color: green;"> </span><span style="color: grey;"></summary></span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">override</span> <span style="color: blue;">void</span> PostExecute()<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">base</span>.PostExecute();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: green;">/*</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">
* Add your code here</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">
*/</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">override</span> <span style="color: blue;">void</span> CreateNewOutputRows()<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: green;">/*</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> Add rows by calling the AddRow method
on the member variable named "<Output Name>Buffer".</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;"> For example, call
MyOutputBuffer.AddRow() if your output was named "MyOutput".</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: green; font-family: "consolas"; font-size: 9.5pt;">
*/</span><span style="font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: #2b91af;">JavaScriptSerializer</span> js = <span style="color: blue;">new</span>
<span style="color: #2b91af;">JavaScriptSerializer</span>();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">byte</span>[] jsonbyte = System.IO.<span style="color: #2b91af;">File</span>.ReadAllBytes(connMgr.ConnectionString);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">string</span> reviewConverted = System.Text.<span style="color: #2b91af;">Encoding</span>.ASCII.GetString(jsonbyte);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: green;">// deserialize the string</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: #2b91af;">jData</span> jdata = js.Deserialize<<span style="color: #2b91af;">jData</span>>(reviewConverted);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">files</span> f <span style="color: blue;">in</span> jdata.rows)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.AddRow();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.id = <span style="color: blue;">string</span>.IsNullOrEmpty(f.id)
? <span style="color: #a31515;">""</span> : f.id.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.dis = <span style="color: blue;">string</span>.IsNullOrEmpty(f.dis)
? <span style="color: #a31515;">""</span> : f.dis.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.siteRef = <span style="color: blue;">string</span>.IsNullOrEmpty(f.siteRef)
? <span style="color: #a31515;">""</span> : f.siteRef.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.assetGmars = <span style="color: blue;">string</span>.IsNullOrEmpty(f.assetGmars)
? <span style="color: #a31515;">""</span> : f.assetGmars.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.assetVfa = <span style="color: blue;">string</span>.IsNullOrEmpty(f.assetVfa)
? <span style="color: #a31515;">""</span> : f.assetVfa.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">
Output0Buffer.equipRef = <span style="color: blue;">string</span>.IsNullOrEmpty(f.equipRef)
? <span style="color: #a31515;">""</span> : f.equipRef.ToString();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">internal</span><span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">class</span> <span style="color: #2b91af;">jData</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: #2b91af;">files</span>[]
rows { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>;
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: "consolas"; font-size: 9.5pt;">internal</span><span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">class</span> <span style="color: #2b91af;">files</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span> id { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>; }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span> dis { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>; }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span> siteRef
{ <span style="color: blue;">get</span>; <span style="color: blue;">set</span>; }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span>
assetGmars { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>;
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span>
assetVfa { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>;
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;"> <span style="color: blue;">public</span> <span style="color: blue;">string</span>
equipRef { <span style="color: blue;">get</span>; <span style="color: blue;">set</span>;
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "consolas"; font-size: 9.5pt;">}<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Hope you like it.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<br />
<div class="MsoNormal">
<b><span style="color: #5b9bd5; font-size: 12.0pt; line-height: 107%;">Posted By: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-75395403230970733582018-06-15T20:31:00.002+05:302018-06-15T20:32:33.225+05:30Spark SQL to join Flat File and JSON File<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">In this article
we are trying to join a Flat File with a JSON file by using SPARK SQL. So were
going to join a structured file with a Semi Structured file. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Hope it will
be interesting.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Flat File and JSON file Meta Data<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;"><br /></span></b></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "arial" , "sans-serif";">JSON file structure:<o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<br /></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">{"empid":101,
"name":"Michael", "salary":3000}<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">{"empid":102,
"name":"Andy", "salary":4500}<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">{"empid":103,
"name":"Justin", "salary":3500}<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">{"empid":104,
"name":"Berta", "salary":4000}<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<br /></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "arial" , "sans-serif";">Flat File Structure:<o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<br /></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">101,Tripura<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">102,West
Bengal<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "arial" , "sans-serif";">103,Bihar<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Scala Code<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;"><br /></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//---------------------------------------</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">// Scala for SPARK to Read JSON File</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">// Join with FLAT file</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">// Creation Date: 05/31/2018</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//-----------------------------------------</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.SparkSession</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.catalyst.encoders.ExpressionEncoder</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> <u>org.apache.spark.sql</u>.Encoder</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> <u>spark</u>.implicits._</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">case</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> </span><b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">class</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> </span><b><span style="color: #a22e00; font-family: "consolas"; font-size: 10.0pt;">Employeestate</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">(</span><span style="color: #0000c0; font-family: "consolas"; font-size: 10.0pt;">empid</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">: Long, </span><span style="color: #0000c0; font-family: "consolas"; font-size: 10.0pt;">state</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">: </span><i><span style="color: #329399; font-family: "consolas"; font-size: 10.0pt;">String</span></i><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//Read JSON File</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> spark =
SparkSession.builder().appName(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"Spark SQL basic example"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">).config(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"spark.some.config.option"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">, </span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"some-value"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">).getOrCreate()</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> <u>spark</u>.implicits._</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> df =
spark.read.json(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"examples/src/main/resources/empsalarydetails.json"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//Making View for JSON file</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">df</span></u><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">.createOrReplaceTempView(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"employee"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//Read FLAT File</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> employeestaeDF =
spark.sparkContext.textFile(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"d:/spark/bin/examples/src/main/resources/employeestate.txt"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">).map(_.split(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">","</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)).map(attributes
=> Employeestate(attributes(</span><span style="color: #c48cff; font-family: "consolas"; font-size: 10.0pt;">0</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">).trim.toInt, attributes(</span><span style="color: #c48cff; font-family: "consolas"; font-size: 10.0pt;">1</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">).trim)).toDF()</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #3f7f5f; font-family: "consolas"; font-size: 10.0pt;">//Making View for FLAT file</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<u><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">employeeDF</span></u><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">.createOrReplaceTempView(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"employeestate"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><u><span style="color: #7f0055; font-family: "consolas"; font-size: 10.0pt;">val</span></u></b><span style="color: black; font-family: "consolas"; font-size: 10.0pt;"> employeeDF =
spark.sql(</span><span style="color: #2a00ff; font-family: "consolas"; font-size: 10.0pt;">"SELECT
employee.empid, employee.name, employee.salary, employeestate.state FROM
employee, employeestate WHERE employee.empid=employeestate.empid;"</span><span style="color: black; font-family: "consolas"; font-size: 10.0pt;">)</span><span style="font-family: "consolas"; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal">
<u><span style="color: black; font-family: "consolas"; font-size: 10.0pt; line-height: 107%;">employeeDF</span></u><span style="color: black; font-family: "consolas"; font-size: 10.0pt; line-height: 107%;">.show<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Output<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyKWaTVs0w6sJX2DYTHvEIla5lrvgU0Z7cEaLTNLSmLBLwn2DiWl25KlZq2HNRJkvKyl9DYYiLfd1p_CpVBpErhyphenhyphenYY5dZr-66DBeWF1IGuFLbQQR7hxDhsKhJ2lqLcCCAl6UNE1VeUI6I/s1600/fig-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="447" data-original-width="650" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyKWaTVs0w6sJX2DYTHvEIla5lrvgU0Z7cEaLTNLSmLBLwn2DiWl25KlZq2HNRJkvKyl9DYYiLfd1p_CpVBpErhyphenhyphenYY5dZr-66DBeWF1IGuFLbQQR7hxDhsKhJ2lqLcCCAl6UNE1VeUI6I/s400/fig-1.JPG" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "sans-serif";">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "arial" , "sans-serif"; mso-themecolor: accent1;">Posted
By: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-74340987058388490722018-06-15T14:11:00.002+05:302018-06-15T14:11:52.588+05:30Twitter Live Streaming with Spark Streaming (Using Scala)<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">In this
article we are trying to demonstrate, How to use Spark streaming with Twitter.
It’s a demo only article and not discussing anything about spark streaming technique.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Create your twitter Apps<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">To stream the
twitter data, you must create twitter apps first.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><a href="https://apps.twitter.com/">https://apps.twitter.com/</a><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">You need 4
key values from “Key and Access Token” of Twitter Apps.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Consumer
Key (API Key)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";"><span style="mso-spacerun: yes;"> </span>Consumer Secret (API Secret)<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">3.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Access
Token<span style="mso-spacerun: yes;"> </span><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">4.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Access
Token Secret <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">5.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Filter
Key<span style="mso-spacerun: yes;"> </span>- is optional, if we need to filter
any specified subject in twitter<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv0a1ALvu50xB0_AZeHuFUuarlR_rwMVWrg4HbStl3XJxWY7wCuvA3ADjwzvscRO2_YW2UJNjQxg1i9brCPGs41knUpwsyTzamwfeqxixMgtjVyIv6aMXRQslWS83mb8NkKG30kVKjxC8/s1600/fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="401" data-original-width="650" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv0a1ALvu50xB0_AZeHuFUuarlR_rwMVWrg4HbStl3XJxWY7wCuvA3ADjwzvscRO2_YW2UJNjQxg1i9brCPGs41knUpwsyTzamwfeqxixMgtjVyIv6aMXRQslWS83mb8NkKG30kVKjxC8/s400/fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJICPWQeiHLPU6sSAfdDydVekl1EL6ey3PRIS_Z2fduEtXVIXgWlSPS2sxpuuz9FbEGkFdaSiL7Vg_r8Ou795LumudLhUO0SnzufIq0D0-Yyw1m3wjFREzF86pszJGJRh_jXJZ7kdyNqY/s1600/fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="287" data-original-width="602" height="190" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJICPWQeiHLPU6sSAfdDydVekl1EL6ey3PRIS_Z2fduEtXVIXgWlSPS2sxpuuz9FbEGkFdaSiL7Vg_r8Ou795LumudLhUO0SnzufIq0D0-Yyw1m3wjFREzF86pszJGJRh_jXJZ7kdyNqY/s400/fig-2.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala IDE build of Eclipse SDK change the pom.xml
dependency<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We need to
add the Dependency on Pom.xml in Scala IDE build of Eclipse SDK<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><dependency><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><span style="mso-tab-count: 1;"> </span><groupId>org.apache.spark</groupId><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><span style="mso-tab-count: 1;"> </span><artifactId>spark-streaming-twitter_2.10</artifactId><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><span style="mso-tab-count: 1;"> </span><version>1.0.0</version><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"></dependency><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Adding twitter4j-core-3.0.6.jar<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We need to
add the reference of “twitter4j-core-3.0.6.jar” file in Scala IDE build of
Eclipse SDK. We can download the “twitter4j-core-3.0.6.jar” jar file from MAVEN
repository.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><a href="https://mvnrepository.com/artifact/org.twitter4j/twitter4j-core/3.0.6">https://mvnrepository.com/artifact/org.twitter4j/twitter4j-core/3.0.6</a><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkpuo3w2WI6DT17l26zGla8RlXw_BS2iD6jNnYZruqwXpXn4ocO2p7AjX2nzpu-6d54UpE673oYJWI0bTEDVOAtIXpByFNXAZt1KeE6OAHA4I9VXXUNUjqZ1Ht29eE8TowVeMx6uNZ-fY/s1600/fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="290" data-original-width="258" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkpuo3w2WI6DT17l26zGla8RlXw_BS2iD6jNnYZruqwXpXn4ocO2p7AjX2nzpu-6d54UpE673oYJWI0bTEDVOAtIXpByFNXAZt1KeE6OAHA4I9VXXUNUjqZ1Ht29eE8TowVeMx6uNZ-fY/s400/fig-3.jpg" width="355" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjufzfnIfzLv3MKQtsLJfB57KzwoB_klgsqrZa8ycOz1VDuCr_tRn3Dnq1UjzCcy16TgXZ6r1zPwM-N0L2XGGVgXQJsWdqNGSo3qsua-hq4OkoTpdjf5IrYhx7CPnrsu87vi6O10oEHuo/s1600/fig-4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="555" data-original-width="713" height="311" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjufzfnIfzLv3MKQtsLJfB57KzwoB_klgsqrZa8ycOz1VDuCr_tRn3Dnq1UjzCcy16TgXZ6r1zPwM-N0L2XGGVgXQJsWdqNGSo3qsua-hq4OkoTpdjf5IrYhx7CPnrsu87vi6O10oEHuo/s400/fig-4.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Create Scala Code <o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipP5jGyhoh3VkBcC7LEfLIcE25VOri7DoWlnOw9eeg4FTioNf7nQMCzXhsVsdIHQMiv11eCVz5ozi6Jn0xo_ghggZl7TaYkB37CIN15cX90XAc9SSq36tWHBP9qYzvf6Hi6H9sbGix2qM/s1600/fig-5.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="704" data-original-width="1366" height="205" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipP5jGyhoh3VkBcC7LEfLIcE25VOri7DoWlnOw9eeg4FTioNf7nQMCzXhsVsdIHQMiv11eCVz5ozi6Jn0xo_ghggZl7TaYkB37CIN15cX90XAc9SSq36tWHBP9qYzvf6Hi6H9sbGix2qM/s400/fig-5.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scala Code Details<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">package</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
com.sqlknowledgebank.spark.sparkstreaming</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
org.apache.spark.SparkConf</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
org.apache.spark.streaming.StreamingContext</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
org.apache.spark.streaming.</span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">Seconds</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
twitter4j.conf.ConfigurationBuilder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
twitter4j.auth.OAuthAuthorization</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> twitter4j.</span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">Status</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">import</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">
org.apache.spark.streaming.twitter.</span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">TwitterUtils</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">object</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">twitterstreaming</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> {</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">def</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">main</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #640067; font-family: Consolas; font-size: 10.0pt;">args</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">: Array[</span><i><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">String</span></i><span style="color: black; font-family: Consolas; font-size: 10.0pt;">]) {</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">if</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> (</span><span style="color: #640067; font-family: Consolas; font-size: 10.0pt;">args</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">length</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;"><</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">4</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">) {</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>System.</span><span style="color: #0000c0; font-family: Consolas; font-size: 10.0pt;">err</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">println</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"Usage: TwitterData
<ConsumerKey><ConsumerSecret><accessToken><accessTokenSecret>"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">+</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"[<filters>]"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>System.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">exit</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">1</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>}</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-tab-count: 1;"> </span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">appName</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"TwitterData"</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">conf</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">new</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> SparkConf()</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">conf</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setAppName</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">appName</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setMaster</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"local[2]"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">ssc</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">new</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> StreamingContext(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">conf</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">Seconds</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">5</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">))</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-tab-count: 1;"> </span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">Array</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">consumerKey</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">consumerSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">accessToken</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">accessTokenSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">) = </span><u><span style="color: #640067; font-family: Consolas; font-size: 10.0pt;">args</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">take</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">4</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">filters</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><u><span style="color: #640067; font-family: Consolas; font-size: 10.0pt;">args</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">takeRight</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #640067; font-family: Consolas; font-size: 10.0pt;">args</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">length</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">-</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #c48cff; font-family: Consolas; font-size: 10.0pt;">4</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">cb</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">new</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> ConfigurationBuilder</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-tab-count: 1;"> </span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">cb</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setDebugEnabled</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">true</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">).</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setOAuthConsumerKey</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">consumerKey</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setOAuthConsumerSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">consumerSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setOAuthAccessToken</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">accessToken</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">setOAuthAccessTokenSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">accessTokenSecret</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-tab-count: 1;"> </span><span style="mso-spacerun: yes;"> </span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">auth</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">new</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> OAuthAuthorization(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">cb</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">build</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><b><span style="color: #7f0055; font-family: Consolas; font-size: 10.0pt;">val</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> </span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">tweets</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> = </span><span style="color: #329399; font-family: Consolas; font-size: 10.0pt;">TwitterUtils</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">createStream</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">ssc</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><b><span style="color: #a22e00; font-family: Consolas; font-size: 10.0pt;">Some</span></b><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">auth</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">), </span><u><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">filters</span></u><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;"> </span><span style="mso-tab-count: 1;"> </span></span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">tweets</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;"> .</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">saveAsTextFiles</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">(</span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"tweets"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">, </span><span style="color: #2a00ff; font-family: Consolas; font-size: 10.0pt;">"json"</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">)</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">ssc</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">start</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">()</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span></span><span style="color: #5e5eff; font-family: Consolas; font-size: 10.0pt;">ssc</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">.</span><span style="color: #4c4c4c; font-family: Consolas; font-size: 10.0pt;">awaitTermination</span><span style="color: black; font-family: Consolas; font-size: 10.0pt;">()</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: black; font-family: Consolas; font-size: 10.0pt;"><span style="mso-spacerun: yes;">
</span>}</span><span style="font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: black; font-family: Consolas; font-size: 10.0pt; line-height: 107%;">}</span><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Passing Command Line Parameters to Run<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimESP3q4E8S911-SHueiEt5vVlYGxHW-70Ay3zGeEhCDAMXFCRVt2Bn2vGpOzurRStgkRlAWrE18fiETxe1VkjRR3x5OJ5NSoYiu0VC3xnJnGavIHy2sX9gLhhrjrVHoRc7yPR3QTp3Tw/s1600/fig-6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="640" data-original-width="1026" height="248" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimESP3q4E8S911-SHueiEt5vVlYGxHW-70Ay3zGeEhCDAMXFCRVt2Bn2vGpOzurRStgkRlAWrE18fiETxe1VkjRR3x5OJ5NSoYiu0VC3xnJnGavIHy2sX9gLhhrjrVHoRc7yPR3QTp3Tw/s400/fig-6.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope it will
be informative.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
By: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com1tag:blogger.com,1999:blog-7468857462999845094.post-3821807382565016062018-04-08T20:19:00.003+05:302018-04-08T20:19:43.982+05:30Download JSON file from Azure Storage and Read it by SSIS – Part 1<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">In
this article we are trying to demonstrate how to download a JSON file from
Azure Storage and read it by SSIS.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">The
solution is in two parts.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Download JSON file from AZURRE BLOB Storage.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Read the JSON file and store it in Relational
Database Table object.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Hope
it will be interesting.<o:p></o:p></span></h3>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><br /></span></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Prepare
Your SSIS<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></h3>
<h2 style="background: white; margin-top: 0in; vertical-align: baseline;">
<span style="color: windowtext; font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; line-height: 107%; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";">If we need to connect with AZURE Storage we need
SSDT 2014 primarily and with this we need </span><span style="color: windowtext; font-family: "Arial","sans-serif"; font-size: 11.0pt; line-height: 107%; mso-fareast-font-family: "Times New Roman";">Microsoft SQL Server 2014 Integration Services Feature Pack
for Azure</span><span style="color: windowtext; font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; line-height: 107%; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";">.<o:p></o:p></span></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif"; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";">We can download it from:<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif"; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";"><a href="https://www.microsoft.com/en-us/download/details.aspx?id=47366">https://www.microsoft.com/en-us/download/details.aspx?id=47366</a><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4TQW2B3Z4v0mLX6eC8RZVWRgE2_b48pmMObfr4ZgP7PRK5olHYkz5OYaNUnoSVgdA6ThPH3NdOX71S6yz85L2u8Nm7ZIEl5XvzVqgoD3AEoVHjVr17wSBWbsLp-jzIiEJzJmLaDHbHqQ/s1600/Fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="312" data-original-width="628" height="197" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4TQW2B3Z4v0mLX6eC8RZVWRgE2_b48pmMObfr4ZgP7PRK5olHYkz5OYaNUnoSVgdA6ThPH3NdOX71S6yz85L2u8Nm7ZIEl5XvzVqgoD3AEoVHjVr17wSBWbsLp-jzIiEJzJmLaDHbHqQ/s400/Fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif"; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman"; mso-no-proof: yes;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif"; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";">When we run the SQL Server 2014
Integration Services Feature Pack, we find the tools to download from Azure
Blob Storage.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif"; mso-bidi-font-weight: bold; mso-fareast-font-family: "Times New Roman";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnHqCwwiLT-2Pv-sisT6ep8v_IZiBz1jL5kqki-MhKnZhr9SMrJIq4qajuDk7XYNbZ669qNwmdJACsT2GNKChT2KPaSz58XM8EiP-EdQAK2LaOCGyiNWYT_o4a3QZYgMwiNdJOlECpQs0/s1600/Fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="300" data-original-width="299" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnHqCwwiLT-2Pv-sisT6ep8v_IZiBz1jL5kqki-MhKnZhr9SMrJIq4qajuDk7XYNbZ669qNwmdJACsT2GNKChT2KPaSz58XM8EiP-EdQAK2LaOCGyiNWYT_o4a3QZYgMwiNdJOlECpQs0/s400/Fig-2.jpg" width="398" /></a></div>
<div class="MsoNormal">
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">JSON
File in Azure Blob Container<br /><br /><o:p></o:p></span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMtfKtesq2QATpR7tdOvWdWdLWA4cZz6rLrWSO00HTNG_1K9oBK8H74iPmmqkQLLvUNXxYAnHrHl8NrqB39vkHbYD6ElgOHJXnmC8zWklUuiGBFe2LBcQzi6uOZ6Yc5ctzuqRDdIrKn3g/s1600/Fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="312" data-original-width="628" height="197" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMtfKtesq2QATpR7tdOvWdWdLWA4cZz6rLrWSO00HTNG_1K9oBK8H74iPmmqkQLLvUNXxYAnHrHl8NrqB39vkHbYD6ElgOHJXnmC8zWklUuiGBFe2LBcQzi6uOZ6Yc5ctzuqRDdIrKn3g/s400/Fig-3.jpg" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Download
JSON File from Azure BLOB Storage<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure BLOB Storage Account and Key<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2uOTWAex3kGMiKvNVMdGztU4vPzwiq5vM_BZvKfVOl15QxSEVZyijLKZb28qsbybTnRvtS9gu8e06bYlyZkWPSNsM9v0pO1LWPvq4fC7WgvrKz8PBJAoRIn4iOYQK9GGYt4nxHnmSnxM/s1600/Fig-4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="275" data-original-width="627" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2uOTWAex3kGMiKvNVMdGztU4vPzwiq5vM_BZvKfVOl15QxSEVZyijLKZb28qsbybTnRvtS9gu8e06bYlyZkWPSNsM9v0pO1LWPvq4fC7WgvrKz8PBJAoRIn4iOYQK9GGYt4nxHnmSnxM/s400/Fig-4.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">SSIS with Azure Blob Download Task<o:p></o:p></span></u></h3>
<div>
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;"><br /></span></u></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTSDFAoZvhn7rK_FWYermcUCLpaNvipDCKjUagxPQxRhZSLlUdmYOVWKmdO89D_DIWpEhqxELe_VALb2o_I7eev6qV7_1UfjPrPNCCJVzCeNsf-PB6in9mfuHpS25Un3VmvpcIW99O13A/s1600/Fig-5.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="215" data-original-width="596" height="143" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTSDFAoZvhn7rK_FWYermcUCLpaNvipDCKjUagxPQxRhZSLlUdmYOVWKmdO89D_DIWpEhqxELe_VALb2o_I7eev6qV7_1UfjPrPNCCJVzCeNsf-PB6in9mfuHpS25Un3VmvpcIW99O13A/s400/Fig-5.jpg" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure Blob Download Task Editor<o:p></o:p></span></u></h3>
<div>
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;"><br /></span></u></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-Q64GWv6IPGjYO0bvJ6T95dNnbbi2YCqpw-Uzb7DrHkK02ALd6r4i7nznRR477ycw3Z0kUPyP-u_BZo55q57ocuuPNEOlv4Q9SH5na5lAGm4azbk3jJJmSR7oG6OLVtTplpbTrn0Rk1g/s1600/Fig-6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="530" data-original-width="624" height="338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-Q64GWv6IPGjYO0bvJ6T95dNnbbi2YCqpw-Uzb7DrHkK02ALd6r4i7nznRR477ycw3Z0kUPyP-u_BZo55q57ocuuPNEOlv4Q9SH5na5lAGm4azbk3jJJmSR7oG6OLVtTplpbTrn0Rk1g/s400/Fig-6.jpg" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure Blob Download Task – Azure Storage
Connection<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpdWOb5uJyp6WhGYWjBHHv67cKXiyg5a4UkleqUyhGtON_RcHp1mWql35Iq6OlWQYlbUJRerRmgMOhP5lhLl3MUu4QuzjaoDtK_GKZP1XAccsMOZ8fayakyhk12v2dpeKX45CrzcLYxjg/s1600/Fig-7.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="362" data-original-width="503" height="287" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpdWOb5uJyp6WhGYWjBHHv67cKXiyg5a4UkleqUyhGtON_RcHp1mWql35Iq6OlWQYlbUJRerRmgMOhP5lhLl3MUu4QuzjaoDtK_GKZP1XAccsMOZ8fayakyhk12v2dpeKX45CrzcLYxjg/s400/Fig-7.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Observing
JSON File in Local Disk<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRc2Ik7rnPzHUCEDt9RWNxR8N7aMYdcho2ahaijnaIZEYsPexQ6-QI1qIcPcAWnJC5k_C8pkcOdGNgFXbpnFFtV8PmizocZftOuI-NzN_HMTMmoWXT46jDoxPKT6wvUs9sps8v-3zE0H0/s1600/Fig-8.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="175" data-original-width="558" height="125" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRc2Ik7rnPzHUCEDt9RWNxR8N7aMYdcho2ahaijnaIZEYsPexQ6-QI1qIcPcAWnJC5k_C8pkcOdGNgFXbpnFFtV8PmizocZftOuI-NzN_HMTMmoWXT46jDoxPKT6wvUs9sps8v-3zE0H0/s400/Fig-8.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: Arial, sans-serif;"><span style="font-size: 14.6667px; font-weight: 400;">Hope you like it. In our next Blog we are moving to "</span></span><span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"> </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-weight: normal; text-indent: -0.25in;">Read the JSON file and store it in Relational Database Table object (Part-2)"</span></h3>
<div>
<span style="font-family: Arial, sans-serif; font-size: 11pt; font-weight: normal; text-indent: -0.25in;"><br /></span></div>
<div>
<span style="font-family: Arial, sans-serif; font-size: 11pt; font-weight: normal; text-indent: -0.25in;"><br /></span></div>
<div>
<span style="font-family: Arial, sans-serif; font-size: 11pt; font-weight: normal; text-indent: -0.25in;"><br /></span></div>
<br />
<br />
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #4f81bd; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-44654893130265756812018-03-02T18:15:00.001+05:302018-03-02T18:15:36.546+05:30Azure Storage<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Before
moving into Azure Data Factory (ADF), we need some general knowledge regarding
Azure storage. The ADF can Extract and Load data from Azure storage. <br />
So, it is necessary to take little bit of knowledge about it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Type of Azure Storage<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><span style="mso-spacerun: yes;"> </span>There are two type of Azure Storage<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 1.0in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 11.0pt; font-weight: normal; mso-bidi-font-family: Wingdings; mso-bidi-font-weight: bold; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Azure Standard Storage<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 1.0in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; font-size: 11.0pt; font-weight: normal; mso-bidi-font-family: Wingdings; mso-bidi-font-weight: bold; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Azure Premium Storage<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQMIl903t_4aNKtLYEiQojc0chhqkhDU0GcXLq8OLkE3TJ7VzJ1ubF0yIYKHJjh1aRCSZy64ko1caaxI6hUaf4Pw3N23XRmckZ-IyZcB4IJg6ilBK81tT9ZpX1Onpq7ZrOaxvyB2T6jow/s1600/Fig-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="386" data-original-width="708" height="217" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQMIl903t_4aNKtLYEiQojc0chhqkhDU0GcXLq8OLkE3TJ7VzJ1ubF0yIYKHJjh1aRCSZy64ko1caaxI6hUaf4Pw3N23XRmckZ-IyZcB4IJg6ilBK81tT9ZpX1Onpq7ZrOaxvyB2T6jow/s400/Fig-1.JPG" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Azure Standard Storage Account<br /><br /><o:p></o:p></span></b></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<b><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Blog Storage<o:p></o:p></span></u></b></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">Blog Storage is basically storage for unstructured data that can include
pictures, videos, music files, documents, raw data, and log data…along with
their meta-data. Blobs are stored in a directory-like structure called a “</span><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">container</span></b><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">”. We can store any
number of blob files up to a total size of 500 TB and we can also apply
security policies. <o:p></o:p></span></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;"><br /></span></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure Table Storage</span></u></b><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;"><o:p></o:p></span></u></div>
<div class="selectionshareable" style="-webkit-text-stroke-width: 0px; background: white; box-sizing: inherit; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: 1rem; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">Table storage, as the name indicates, is preferred for tabular data,
which is ideal for key-value NoSQL data storage. Table Storage is massively scalable
and extremely easy to use. <o:p></o:p></span></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure File Storage</span></u></b><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;"><o:p></o:p></span></u></div>
<div class="selectionshareable" style="-webkit-text-stroke-width: 0px; background: white; box-sizing: inherit; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: 1rem; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">Azure File Storage is meant for legacy applications. Azure VMs and
services share their data via mounted file shares, while on-premise
applications access the files using the File Service REST API. Azure File
Storage offers file shares in the cloud using the standard SMB protocol and
supports both SMB 3.0 and SMB 2.1.<o:p></o:p></span></div>
<div class="selectionshareable" style="-webkit-text-stroke-width: 0px; background: white; box-sizing: inherit; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: 1rem; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Azure Queue Storage</span></u></b><u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;"><o:p></o:p></span></u></div>
<div class="selectionshareable" style="-webkit-text-stroke-width: 0px; background: white; box-sizing: inherit; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: 1rem; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">The Queue Storage service is used to exchange messages between components
either in the cloud or on-premise (compare to Amazon’s SQS). We can store large
numbers of messages to be shared between independent components of applications
and communicated asynchronously via HTTP or HTTPS. Typical use cases of Queue
Storage include processing backlog messages or exchanging messages between
Azure Web roles and Worker roles.<o:p></o:p></span></div>
<div class="selectionshareable" style="background: white; margin-top: 0in;">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Azure Premium Storage account<br /><br /><o:p></o:p></span></b></div>
<div class="selectionshareable" style="-webkit-text-stroke-width: 0px; background: white; box-sizing: inherit; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: 1rem; margin-top: 0in; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: bold;">The Azure Premium Storage service is the most recent storage offering
from Microsoft, in which data are stored in Solid State Drives (SSDs) for
better IO and throughput. Premium storage only supports Page Blobs.<o:p></o:p></span></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Hope
you like it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #5b9bd5; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com10tag:blogger.com,1999:blog-7468857462999845094.post-67112393547353100182018-02-24T22:03:00.003+05:302018-02-24T22:03:28.267+05:30Azure Data Lake – U-SQL – Using C# Code Behind<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">In
this article we are trying to discuss about Using C# Code behind in U-SQL
script. As we look that every U-SQL script has a C# code behind.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP1KIjgyY66yUqRvKre54EnuMuwRqoVr59ahzcvJU6MRzP8h_sswFb1u87N-_EMdYyEW6-6cR1qRMMVaX77Oj4ZDCDyVf6kFZv8f6-kGMMQ_Mn-5glbrg1Xt8owZvGX-k9jFz8WvHHFk4/s1600/fig-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="290" data-original-width="353" height="327" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhP1KIjgyY66yUqRvKre54EnuMuwRqoVr59ahzcvJU6MRzP8h_sswFb1u87N-_EMdYyEW6-6cR1qRMMVaX77Oj4ZDCDyVf6kFZv8f6-kGMMQ_Mn-5glbrg1Xt8owZvGX-k9jFz8WvHHFk4/s400/fig-1.JPG" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Now
the question is why we are going to use this C# code behind as we can create
function, stored procedure and other thing successfully in U-SQL. The answer is
quite simple. If we want to use the power of C# and the entire library related
to it, we can use the C# code behind. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><span style="mso-spacerun: yes;"> </span>For an example, we need to create a complex scalar
value function and using C# it is quite easy by using it’s math library inbuilt
functionality. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Case Study<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><span style="mso-spacerun: yes;"> </span>To understand the C# code behind, we are not taking
any complex example. Here we have a CSV file, that have “StudentID”, “StidentName”
,“Marks1”, “Marks2” and “Marks3”. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">We
are going to retrieve information from CSV file and try to put the information
to another output CSV file. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">We
are doing little transformation work by adding “Marks1”, “Marks2” and “Marks3”
and give it a “Total Marks”.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">We
are going to use C# code behind by creating a function name “GetTotalMarks”. It
takes three input marks and returns the total of three input marks.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOMgf6USzxPcFr9OwEfZfmyVVnlOQBRPzAq9nmCP8DtD5Z-cjgie1YHqzuHz6DItAcjmdvzwHb_WmGD_6g3KXakK0rjj2RpWOV26p6EjfQpBf9GNEsQiWxPonhIsu3sBwICOI-0cfBWu0/s1600/fig-2.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="110" data-original-width="386" height="113" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOMgf6USzxPcFr9OwEfZfmyVVnlOQBRPzAq9nmCP8DtD5Z-cjgie1YHqzuHz6DItAcjmdvzwHb_WmGD_6g3KXakK0rjj2RpWOV26p6EjfQpBf9GNEsQiWxPonhIsu3sBwICOI-0cfBWu0/s400/fig-2.JPG" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">C# Code
Behind<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> Microsoft.Analytics.Interfaces;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> Microsoft.Analytics.Types.Sql;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> System;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> System.Collections.Generic;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> System.IO;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> System.Linq;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> System.Text;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">namespace</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> <span style="background: yellow; mso-highlight: yellow;">TestApplication</span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">{<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">public</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">static</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">class</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: aqua; color: #2b91af; font-family: Consolas; font-size: 9.5pt; mso-highlight: aqua;">StudentRecor</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>{<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">public</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">static</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> Double <span style="background: fuchsia; mso-highlight: fuchsia;">GetTotalMarks</span>(</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> marks_1, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> marks_2, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> marks_3)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>{<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">return</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> marks_1 + marks_2 + marks_3;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>}<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>}<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">}<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">U-SQL
Script<o:p></o:p></span></h3>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StudentID</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StudentName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks1</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks2</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks3</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/Input-1/StudentRecords.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StudentID</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StudentName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks1</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks2</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks3</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">TestApplication</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: aqua; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: aqua;">StudentRecor</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">GetTotalMarks</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks1</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks2</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Marks3</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">TotalMarks</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/output/Output-1/StudentResult.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Please
look at the calling of Function in U-SQL code. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">It
is :<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt;"><Name Space
Name> . <Class Name>.<Function Name><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt;">Job Graph:<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSNJvL4mlV30JZ9-gqtgbx0WffH9GHmQhM_6t3kO9EjOaZpJj-9pKy3-VKl17D0H_T08H7rIwVjCpiCzR6gBNOS7Ib8OqEmiH0TwbHt87QdlfYTLcxVu27mbfuvhAnoZpru1auhaEhFyw/s1600/fig-4.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="392" data-original-width="379" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSNJvL4mlV30JZ9-gqtgbx0WffH9GHmQhM_6t3kO9EjOaZpJj-9pKy3-VKl17D0H_T08H7rIwVjCpiCzR6gBNOS7Ib8OqEmiH0TwbHt87QdlfYTLcxVu27mbfuvhAnoZpru1auhaEhFyw/s400/fig-4.JPG" width="386" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Output
File<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPr2n8EVOiB4fdFf9h_oF5_yqLrcHeup-rsmlcRLTrqencF6G-PyGRSHhZVJv7aD1UY1QzwUMY7VO2Nax3CwUk7fw85VakpIAXcJROIvzcukUP-_i3ACXWkXJklRtR1G7j8TYiou50BvM/s1600/fig-3.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="85" data-original-width="233" height="145" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPr2n8EVOiB4fdFf9h_oF5_yqLrcHeup-rsmlcRLTrqencF6G-PyGRSHhZVJv7aD1UY1QzwUMY7VO2Nax3CwUk7fw85VakpIAXcJROIvzcukUP-_i3ACXWkXJklRtR1G7j8TYiou50BvM/s400/fig-3.JPG" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;">Hope
you like it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p> </o:p></span></h3>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p><br /></o:p></span></div>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p><br /></o:p></span></div>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-bidi-font-weight: bold;"><o:p><br /></o:p></span></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #5b9bd5; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com7tag:blogger.com,1999:blog-7468857462999845094.post-65096748819491857902018-02-23T17:06:00.002+05:302018-02-23T17:06:21.122+05:30Azure Data Lake – U-SQL – Lab Session -1<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<b style="background-color: transparent;"><span style="font-family: "Arial",sans-serif; font-size: 12.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;">Here in this
article, we are moving with a simple lab session which include all the article
that we published before. By this article we get a good concept to work with
U-SQL.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial",sans-serif; font-size: 12.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Case study<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;">We have four
CSV files and one U-SQL Table object named Sales Order<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial",sans-serif;">Customer.csv<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial",sans-serif;">ItemGroupMaster.csv<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial",sans-serif;">ItemMaster.csv<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Courier New"; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial",sans-serif;">ItemStockMaster.csv
<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;">Meta data information
are given bellow<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><b><u>Sales Order
Table</u></b><o:p></o:p></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
<td style="background: #C5E0B3; border-left: none; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Data
Type<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">SrlNo<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">OrderNumber<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">CustID<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">ItemCode<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">SaleQty<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">int<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><b><u>ItemMaster.csv </u></b><o:p></o:p></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
<td style="background: #C5E0B3; border-left: none; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Data
Type<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
ItemCode<span style="font-family: "Arial",sans-serif;"><o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
ItemName<o:p></o:p></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
GroupCode<o:p></o:p></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><b><u>ItemGroupMaster.csv</u></b><o:p></o:p></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
<td style="background: #C5E0B3; border-left: none; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Data
Type<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
ItemGroupCode<span style="font-family: "Arial",sans-serif;"><o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
ItemGroupName<o:p></o:p></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><b><u>Customer.csv</u></b><o:p></o:p></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
<td style="background: #C5E0B3; border-left: none; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Data
Type<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">CustomerCode<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">CustomerName<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">String<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><b><u>ItemStockMaster.csv</u></b><o:p></o:p></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
<td style="background: #C5E0B3; border-left: none; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Data
Type<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">ItemCode<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 130.25pt;" valign="top" width="174">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">StockQty<o:p></o:p></span></div>
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 81.0pt;" valign="top" width="108">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">Int<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;">The output we
needed by combining all those things are:<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableGrid" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="background: #C5E0B3; border: solid windowtext 1.0pt; mso-background-themecolor: accent6; mso-background-themetint: 102; mso-border-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<b><span style="font-family: "Arial",sans-serif;">Column
Name<o:p></o:p></span></b></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">ItemCode<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">ItemName<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">ItemGroupName<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">StockQty<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">OrderNumber<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">CustomerID<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">CustomerName<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">SaleQty<o:p></o:p></span></div>
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0in 5.4pt 0in 5.4pt; width: 211.25pt;" valign="top" width="282">
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial",sans-serif;">RemainingStockQty<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial",sans-serif; font-size: 12.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">U-SQL Statement<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial",sans-serif; font-size: 12.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">; <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@itemmaster</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">=<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">GroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"D:/USQL-Project/Project-1/Input/ItemMaster.csv"</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@joining</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">b</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemGroupName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">c</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StockQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">OrderNumber</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustID</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">e</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustomerName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">SaleQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">c</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StockQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> - </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">SaleQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">stockQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">GroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@itemmaster</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> ) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INNER</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">JOIN</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemGroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemGroupName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemGroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> ) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">b</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">ON</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">GroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> == </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">b</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemGroupCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INNER</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">JOIN</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">StockQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemStock</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> ) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">c</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">ON</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> == </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">c</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">LEFT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">JOIN</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">OrderNumber</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustID</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: purple; font-family: Consolas; font-size: 9.5pt;">SUM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">(</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">SaleQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">SaleQty</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">salesOrder</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">GROUP</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">BY</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">OrderNumber</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustID</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> ) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">ON</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> == </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">LEFT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">JOIN</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> (<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustomerId</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustomerName</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"D:/USQL-Project/Project-1/Input/Customer.csv"</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">()<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> ) </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">e</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">ON</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">d</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustID</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> == </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">e</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">CustomerId</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">; <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@joining</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"D:/USQL-Project/Project-1/Output/Result.csv"</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> </span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">.</span></b><b><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial",sans-serif;"><br /></span></div>
<br />
<div class="MsoNormal">
<b><span style="color: #5b9bd5; font-family: "Arial",sans-serif; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-52816840363846302572018-02-22T14:41:00.000+05:302018-02-22T14:41:19.258+05:30Azure Data Lake – U-SQL – Join CSV and DB Table<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In my previous article we are going to combined
read two CSV file and making grouping and finding the total quantity of Item.<o:p></o:p></span></h3>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<span style="font-family: "Arial","sans-serif";">Azure Data
Lake – U-SQL – Using Wildcard or Virtual Columns<br />
<a href="http://sqlknowledgebank.blogspot.in/2018/02/azure-data-lake-u-sql-using-wildcard-or.html">http://sqlknowledgebank.blogspot.in/2018/02/azure-data-lake-u-sql-using-wildcard-or.html</a><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<span style="font-family: "Arial","sans-serif";">Another
article that I create database, schema and table object.<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<span style="font-family: "Arial","sans-serif";">Azure Data
Lake – U-SQL – Creating Database and DB Object<br />
<a href="http://sqlknowledgebank.blogspot.in/2018/02/azure-data-lake-u-sql-creating-database.html">http://sqlknowledgebank.blogspot.in/2018/02/azure-data-lake-u-sql-creating-database.html</a><br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<span style="font-family: "Arial","sans-serif";">Now it is
the time to combine them both. Hope it will be interesting.<o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt; mso-outline-level: 3;">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scenario<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Here we are going to combine a CSV file with a
Database Table object. I mean to say that joining a CSV file with Database
table object and retrieve some common information from both the CSV file and
Table object.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFf_hfGBBVYU1TN4pediNVIxECx6SMvvs7emOjZe-KJLiX3FVqMyEfIJkGI_MC6RsJxv5cL2rG24bCOE-kDNy_E6H2P3iVG56CpaHLwvstbFTRNTCDBxWx-lWFfrNwFT9_v6mStYtHt68/s1600/Fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="275" data-original-width="634" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFf_hfGBBVYU1TN4pediNVIxECx6SMvvs7emOjZe-KJLiX3FVqMyEfIJkGI_MC6RsJxv5cL2rG24bCOE-kDNy_E6H2P3iVG56CpaHLwvstbFTRNTCDBxWx-lWFfrNwFT9_v6mStYtHt68/s400/Fig-1.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">U-SQL
to Join</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// DATABASE keyword is optional</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">; </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// SCHEMA keyword is
required for schemas</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">filename</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/Input-1/{filename:}.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">a</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">b</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">b</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">Total</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">a</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">a</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INNER</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">JOIN</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>(<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: purple; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">Total</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: lime; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: lime;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span>) </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">b</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">ON</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">a</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;"> == </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">b</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: fuchsia; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: fuchsia;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/output/Output-1/Result.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Job Graph:<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfyCD7yevoBqaZZmUMDy_GHplkIk-isQ_0vE62FU9D9JNDw_OAO9gzuKi18Reu64tdT8zpSdCdXS-zzNQepYDHQzKNGJ9z2wL9xT0NwpNTSRZfLNEP2MWWuqJHJK18qclD5h78a8jPQRs/s1600/Fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="343" data-original-width="644" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfyCD7yevoBqaZZmUMDy_GHplkIk-isQ_0vE62FU9D9JNDw_OAO9gzuKi18Reu64tdT8zpSdCdXS-zzNQepYDHQzKNGJ9z2wL9xT0NwpNTSRZfLNEP2MWWuqJHJK18qclD5h78a8jPQRs/s400/Fig-2.jpg" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Type of
Join </span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .1in; margin-left: 0in; margin-right: 0in; margin-top: .1in;">
<span style="font-family: "Arial","sans-serif";">U-SQL supports a number of join types – some may well be
familiar to you, others may not.<o:p></o:p></span></div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->INNER
JOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->FULL
OUTER JOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->LEFT
OUTER JOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->RIGHT
OUTER JOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->CROSS
JOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->LEFT
SEMIJOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->RIGHT
SEMIJOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->LEFT
ANTISEMIJOIN</div>
<div class="MsoNoSpacing" style="margin-left: 1.5in; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;"><span style="mso-list: Ignore;">ü<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->RIGHT
ANTISEMIJOIN</div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Hope you like it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #5b9bd5; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-91014507084695237392018-02-22T10:02:00.000+05:302018-02-22T10:02:05.353+05:30Azure Data Lake – U-SQL – Creating Database and DB Object<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss about creation
of Database and DB Object. Hope it will be informative.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scenario<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">What we are doing in this article is mentioned bellow:<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Create a Database in the name of “MyUSQLDb”.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Create a Schema in the name of “AbcPvtLtd”.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">3.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Create a Table in the name of “ItemRate”.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;"><span style="mso-list: Ignore;">4.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Insert some records in the table objects.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxcs7AXaQs6cpU5z4HMbLh1oR971N8l3PO7yKFEDY16HpBE5XBMeDgL2LZxBePYxquJvbADyHjPzJhdA_rcdudNfYSJBF0Qy3K4K3ck00FpqmGoOknab0q_SXQoiCR6E1-tIwFwZNIP3U/s1600/fig-0.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="481" data-original-width="380" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxcs7AXaQs6cpU5z4HMbLh1oR971N8l3PO7yKFEDY16HpBE5XBMeDgL2LZxBePYxquJvbADyHjPzJhdA_rcdudNfYSJBF0Qy3K4K3ck00FpqmGoOknab0q_SXQoiCR6E1-tIwFwZNIP3U/s640/fig-0.JPG" width="504" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">CREATE Database<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">To Create Database <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">CREATE</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">IF</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">NOT</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXISTS</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">To Drop a Database<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DROP</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">IF</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXISTS</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">CREATE
Schema<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">To Create Schema<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">CREATE</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">IF</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">NOT</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXISTS</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">To Drop Schema<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DROP</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">IF</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">NOT</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXISTS</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">CREATE
Table Object and Index<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// DATABASE keyword is optional</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">; </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// SCHEMA keyword is
required for schemas</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">CREATE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TABLE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">IF</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">NOT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXISTS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INDEX</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">idx_ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">CLUSTERED</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> (</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DISTRIBUTED</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">HASH</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> (</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">)<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;">);</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Insert
Records in Table<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">DATABASE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">MyUSQLDb</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// DATABASE keyword is optional</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USE</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SCHEMA</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">AbcPvtLtd</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">; </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: green; font-family: Consolas; font-size: 9.5pt;">// SCHEMA keyword is
required for schemas</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INSERT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INTO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">VALUES</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">101</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">210</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">);<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INSERT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INTO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">VALUES</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">102</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">120</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">);<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INSERT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">INTO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemRate</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">VALUES</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">103</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #003200; font-family: Consolas; font-size: 9.5pt;">112</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;">);</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Job Graph:<o:p></o:p></span></u></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmyNF2_3VqbES7-SbJ-njpyU_2o4QdPAMMjRKcnqAnXGWCW65WZZkmMxA82y0czqBBBKlXrnsSsDu6OLuNVY0kXSik0Aaat6wIyAXfRqOEIxFwWAjIANJqRNrOZUEryeaXGZxncgLKX1A/s1600/fig-1.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="395" data-original-width="655" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmyNF2_3VqbES7-SbJ-njpyU_2o4QdPAMMjRKcnqAnXGWCW65WZZkmMxA82y0czqBBBKlXrnsSsDu6OLuNVY0kXSik0Aaat6wIyAXfRqOEIxFwWAjIANJqRNrOZUEryeaXGZxncgLKX1A/s400/fig-1.JPG" width="400" /></a></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Hope you like it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #5b9bd5; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com5tag:blogger.com,1999:blog-7468857462999845094.post-7305233279119018682018-02-21T20:46:00.004+05:302018-02-21T20:46:52.463+05:30Azure Data Lake – U-SQL – Using Wildcard or Virtual Columns <div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss about
Wildcard character and Virtual Columns.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scenario<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">We have two CSV files named “ItemDetails_1.csv”
and “ItemDetails_2.csv”. Both CSV file have Item code, Item Name and Quantity.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">We want to grouping by Item Code and Item Name
and what the Summation of Item Quantity.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg86hb-vv_p4D4NFYfaW9BegKWKiF6B8AyxaGCh0vEt8aA0J3PZNyPA4i89OQ34eFHtSYQcwV25nL3JYuOxNeT1r6YiJ9z1IZvYbbTrlLS5asej_n-ijgWgto59jrk8388-R3KOJMToGsk/s1600/Fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="184" data-original-width="436" height="168" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg86hb-vv_p4D4NFYfaW9BegKWKiF6B8AyxaGCh0vEt8aA0J3PZNyPA4i89OQ34eFHtSYQcwV25nL3JYuOxNeT1r6YiJ9z1IZvYbbTrlLS5asej_n-ijgWgto59jrk8388-R3KOJMToGsk/s400/Fig-1.jpg" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Please note that here the input file name is
mostly similar. So, we are using wildcard character or Virtual Columns.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Using Wildcard Character<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/Input-1<span style="background: yellow; mso-highlight: yellow;">/{*}.csv"</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: purple; font-family: Consolas; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Total</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/output/Output-1/Result.csv"<br />
</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();</span></b><b><span style="font-family: "Arial","sans-serif";"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">It takes any CSV file as input from location <br />
</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">C:/Users/Joydeep/AppData/Local/USQLDataRoot/Input-1/</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Job Graph:<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxpm_qRwolj8EgJjgCnjk0H2NncMB8T7zwUJAdrD1MotWUiZ8En4-zhupWR-J_lSs3a2Glgul2utegUkLn9JVlJOpCT1ImfFgu-4C9JxkYW3JoeR9DI_AlZhYEs5_U55LQgXbP_TzqK3o/s1600/Fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="398" data-original-width="441" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxpm_qRwolj8EgJjgCnjk0H2NncMB8T7zwUJAdrD1MotWUiZ8En4-zhupWR-J_lSs3a2Glgul2utegUkLn9JVlJOpCT1ImfFgu-4C9JxkYW3JoeR9DI_AlZhYEs5_U55LQgXbP_TzqK3o/s400/Fig-2.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Using
Virtual Columns<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">filename</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/Input-1/<span style="background: yellow; mso-highlight: yellow;">{filename:}.csv</span>"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: purple; font-family: Consolas; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Qty</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Total</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemCode</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ItemName</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@filtering</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"C:/Users/Joydeep/AppData/Local/USQLDataRoot/output/Output-1/Result.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();</span><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Job Graph:<o:p></o:p></span></u></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRX_YRdQQgRZAj5mXNaQ9dX0mWpowJUX7YLUGovJ3BbElQfOD9ryP2TrGzvtaBgVqwBvxd_q44MYK7x4bfdkRKaWGyhupPKCkU_CFOP6xQrVZKqo6xWNj_gGU5YBUF7jM55A_DxjaMq6E/s1600/Fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="393" data-original-width="496" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRX_YRdQQgRZAj5mXNaQ9dX0mWpowJUX7YLUGovJ3BbElQfOD9ryP2TrGzvtaBgVqwBvxd_q44MYK7x4bfdkRKaWGyhupPKCkU_CFOP6xQrVZKqo6xWNj_gGU5YBUF7jM55A_DxjaMq6E/s400/Fig-3.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Hope you like it.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="color: #5b9bd5; font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></h3>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-62674483342234328622018-02-18T20:17:00.000+05:302018-02-18T20:17:06.076+05:30Azure Data Lake – U-SQL – A small Hands-on with VS 2017<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss install
Visual Studio 2017 that is integrated with Azure Data Lake and trying to
execute a small U-SQL script as testing purpose.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Hope it will be informative and influence you
to start your first U-SQL script.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Installing Visual Studio 2017<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">We are using visual studio 2017 community
edition for that. It’s free.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><a href="https://www.visualstudio.com/vs/community/">https://www.visualstudio.com/vs/community/</a><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBJwn-idVHouu8_OeBCK4EPJn2PfeCoqS84IYxG0fPZE1NBba8rSk_Xn9pQuJLNMe0sDzFkC-00tcM5TI2qI6mPIg0V5XXNpxYg8K5DeFDT_0kQuRnldoX_7o09aPu2LYupbBeB50YROc/s1600/Fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="360" data-original-width="729" height="197" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBJwn-idVHouu8_OeBCK4EPJn2PfeCoqS84IYxG0fPZE1NBba8rSk_Xn9pQuJLNMe0sDzFkC-00tcM5TI2qI6mPIg0V5XXNpxYg8K5DeFDT_0kQuRnldoX_7o09aPu2LYupbBeB50YROc/s400/Fig-1.jpg" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Now when we are going to run the U-SQL Script
(if OS is Win 7), probably get an compile error.<o:p></o:p></span></h3>
<div>
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNoSpacing">
<b>“</b><b style="mso-bidi-font-weight: normal;"><span style="color: red;">Error E_RUNTIME_USER_LOADENGINEASSEMBLY: .Net cannot load
runtime assembly 'ScopeEngineManaged.dll'.<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: red;">Description:<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: red;">Could not load file or assembly 'ScopeEngineManaged.dll' or
one of its dependencies. The specified module could not be found.<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: red;">Resolution:<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: red;">Ensure the file exists and that all dependencies are
available, including ScopeEngine.dll. <o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: red;">This runtime assembly requires a 64-bit process and depends
upon the VC14 CRT runtime being installed on the local machine.<span style="mso-tab-count: 1;"> </span>TestApplication<span style="mso-tab-count: 1;"> </span>E:\u-SQL\TestApplication\TestApplication\Script.usql</span></b>”</div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">To solve
this we need to run another patch called “Universal C for Runtime for Windows
(64 bit)”. <o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<a href="http://www.microsoft.com/downloads/details.aspx?familyid=cdd5866a-43b3-409e-8fd7-e37b0a9ccc63"><span style="background: white; border: none windowtext 1.0pt; color: #005999; font-family: "Arial","sans-serif"; font-size: 11.5pt; mso-border-alt: none windowtext 0in; padding: 0in;">http://www.microsoft.com/downloads/details.aspx?familyid=cdd5866a-43b3-409e-8fd7-e37b0a9ccc63</span></a></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcpyacVeW4fajE6diX6Tv19iyaauE4yPLjQDNgwoWjU0AYge1pDJ0UTwOdI4qYDcSGU4OrMFw7uy9QwUfx1_Yjz8hYse7CaUQN8qqe1gXEYs60gOmgrjoiiE-9OaUG4fLt-eINz5pGyAo/s1600/Fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="456" data-original-width="943" height="192" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcpyacVeW4fajE6diX6Tv19iyaauE4yPLjQDNgwoWjU0AYge1pDJ0UTwOdI4qYDcSGU4OrMFw7uy9QwUfx1_Yjz8hYse7CaUQN8qqe1gXEYs60gOmgrjoiiE-9OaUG4fLt-eINz5pGyAo/s400/Fig-2.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Hands-on
Practice for Testing<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Finding
the Root directory for Input file<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b style="background-color: orange;"><span style="font-family: "Arial","sans-serif";">Tools </span><span style="font-family: Wingdings; mso-ascii-font-family: Arial; mso-bidi-font-family: Arial; mso-char-type: symbol; mso-hansi-font-family: Arial; mso-symbol-font-family: Wingdings;"><span style="mso-char-type: symbol; mso-symbol-font-family: Wingdings;">à</span></span></b><span style="font-family: "Arial","sans-serif";"><b style="background-color: orange;"> Options</b><o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI8JVG2BK-0ULuAcZuehZJxlg2tF4OG2ITyeeOIGqseyckrdTcNlxqTsv0Vutys7Qv2TdYWTAMyM6rEIF17hjGIfsHa47RRNwuEqEfseV8yqM0kR2SDz0-dv_MCjfBlJ-rPp73Gx0ai8k/s1600/Fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="395" data-original-width="722" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI8JVG2BK-0ULuAcZuehZJxlg2tF4OG2ITyeeOIGqseyckrdTcNlxqTsv0Vutys7Qv2TdYWTAMyM6rEIF17hjGIfsHa47RRNwuEqEfseV8yqM0kR2SDz0-dv_MCjfBlJ-rPp73Gx0ai8k/s400/Fig-3.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">In my case
the Data Root directory is: <o:p></o:p></span></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";">C:\Users\Joydeep\AppData\Local\USQLDataRoot<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">In my case
the Output Directory is:<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";">C:\Users\Joydeep\AppData\Local\USQLDataRoot\Output<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Now
download a TSV file from the given link<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #4184c5; font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"><a href="https://raw.githubusercontent.com/Azure/usql/master/Examples">https://raw.githubusercontent.com/Azure/usql/master/Examples</a>/Samples/Data/SearchLog.tsv</span><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: #4184c5; font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"><br /></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">*** Put the
file in Root directory (just copy and paste)<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOgbb8N_Y8YOzPINQNlkG3dTY2xb-xGbKO_iMTStoZVXKuqwLQP2Zmbhu7QyJ0LtFdKx59Md7k5gWKP0gJmQ20rZkEpyJPdrvGiu-XUpQJ8StFmiDh7OVsW2XQkJyKH5HWF45sor_Tr0/s1600/Fig-4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="233" data-original-width="665" height="140" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUOgbb8N_Y8YOzPINQNlkG3dTY2xb-xGbKO_iMTStoZVXKuqwLQP2Zmbhu7QyJ0LtFdKx59Md7k5gWKP0gJmQ20rZkEpyJPdrvGiu-XUpQJ8StFmiDh7OVsW2XQkJyKH5HWF45sor_Tr0/s400/Fig-4.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Now open
the U-SQL Editor and put the U-SQL Script<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #006060; font-family: Consolas; font-size: 9.5pt;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"<span style="background: yellow; mso-highlight: yellow;">C:/Users/Joydeep/AppData/Local/USQLDataRoot/SearchLog.tsv</span>"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #a05000; font-family: Consolas; font-size: 9.5pt;">"<span style="background: yellow; mso-highlight: yellow;">C:/Users/Joydeep/AppData/Local/USQLDataRoot/output/SearchLog-first-u-sql.csv</span>"</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #0000d0; font-family: Consolas; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: #505000; font-family: Consolas; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Note: Input
and output directory is marked by yellow marking.<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Now run
the script by clicking Submit button in U-SQL Editor<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_MRB_BwWH6k3layf-swBrL1cs1SI_9dbexEG5OcfuY7R6kwt-hiYla3lVfjZODW0ZsMzt88C9V-O_wbMH9eUDI39kCM8gbOpOuvBg3RYPhJ5FmHmoAWFQ3g9GW0THr_Wv1l60IlpofO8/s1600/Fig-5.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="286" data-original-width="640" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_MRB_BwWH6k3layf-swBrL1cs1SI_9dbexEG5OcfuY7R6kwt-hiYla3lVfjZODW0ZsMzt88C9V-O_wbMH9eUDI39kCM8gbOpOuvBg3RYPhJ5FmHmoAWFQ3g9GW0THr_Wv1l60IlpofO8/s400/Fig-5.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Observing
Output<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbvKb-MOXTOAPm1M4W7-0dD0IY9-TKY-OAO9R3Qr-rDxKDs1LSDyZd_YkBG_FRZz5-AY59E27S9QRbjTEsUmWU6pU6xS72PqPoDSHoLOKgeK_iehJ7I3LZ9Er5Cm_ri6vb72zPky_FpVg/s1600/Fig-6.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="208" data-original-width="412" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbvKb-MOXTOAPm1M4W7-0dD0IY9-TKY-OAO9R3Qr-rDxKDs1LSDyZd_YkBG_FRZz5-AY59E27S9QRbjTEsUmWU6pU6xS72PqPoDSHoLOKgeK_iehJ7I3LZ9Er5Cm_ri6vb72zPky_FpVg/s400/Fig-6.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwIprQphJMNk9Dwbc7z7OwrYNlPkTe7AcPcyyPkQ8u3VsuGN6OMBtTk5hC9Fdj9X2QZ2OUg04UKGzRxcXOGvpSVqlVzQIboqpCKrsauea6B8Jdy5RRnmVBVzXFAImbKuPP6_LQ0NTQniE/s1600/Fig-7.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="375" data-original-width="1097" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwIprQphJMNk9Dwbc7z7OwrYNlPkTe7AcPcyyPkQ8u3VsuGN6OMBtTk5hC9Fdj9X2QZ2OUg04UKGzRxcXOGvpSVqlVzQIboqpCKrsauea6B8Jdy5RRnmVBVzXFAImbKuPP6_LQ0NTQniE/s400/Fig-7.jpg" width="400" /></a></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";">Hope you
like it.<o:p></o:p></span></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<span style="font-family: "Arial","sans-serif";"><o:p><br /></o:p></span></div>
<div class="MsoNoSpacing">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<div class="MsoNoSpacing">
<br /></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com5tag:blogger.com,1999:blog-7468857462999845094.post-34991168336279030252018-02-16T20:46:00.000+05:302018-02-16T20:51:38.569+05:30Azure Data Lake – U-SQL – Built-in UDOs<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Introduction</span></b><br />
<div class="separator" style="clear: both; text-align: center;">
<b><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_vXZeygQGqDxf6HSnA3AnMEc1oV4Rhvuzw9v_HevJiOoYSiDtOA8eOXUFSG3F-2cYQR_OYFEAB2kMon8S78NoLmk03K6syYPzWF1VEAZ0rhdPUD1KeFb58bqFRlzggXNvXgUjmyd_Zug/s1600/logo.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="213" data-original-width="211" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_vXZeygQGqDxf6HSnA3AnMEc1oV4Rhvuzw9v_HevJiOoYSiDtOA8eOXUFSG3F-2cYQR_OYFEAB2kMon8S78NoLmk03K6syYPzWF1VEAZ0rhdPUD1KeFb58bqFRlzggXNvXgUjmyd_Zug/s200/logo.jpg" width="198" /></a></span></b></div>
</div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;">In this article we are trying to discuss about Built-in
UDOs. That we used on Extraction and Output forms. As we in our previous
article we are frequently used<div class="separator" style="clear: both; text-align: center;">
</div>
<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Extractors</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Tsv</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();</span><span style="color: black; font-family: "consolas"; font-size: 9.5pt;"><br />
</span><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Outputters</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Csv</span><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();</span><span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;"><o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;">These UDOs is used for some common data format.
Microsoft is adding more UDOs in feature. <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Type of UDOs<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">We have two types of UDOs.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableLightShading" style="border-collapse: collapse; border: none; mso-border-bottom-alt: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-alt: solid black 1.0pt; mso-border-top-themecolor: text1; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: -1;">
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-yfti-cnfc: 5;">
<span style="color: black; font-family: "segoe ui semibold" , "sans-serif"; font-size: 11.5pt;">UDO<b><o:p></o:p></b></span></div>
</td>
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-yfti-cnfc: 1;">
<span style="color: black; font-family: "segoe ui semibold" , "sans-serif"; font-size: 11.5pt;">Description<b><o:p></o:p></b></span></div>
</td>
</tr>
<tr style="box-sizing: border-box; mso-yfti-irow: 0;">
<td data-th="UDO" style="background: silver; border: none; box-sizing: border-box; min-width: 80px; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-yfti-cnfc: 68;">
<b><span style="color: #333333; font-family: "segoe ui" , "sans-serif"; font-size: 11.5pt;">Extractors<o:p></o:p></span></b></div>
</td>
<td data-th="Description" style="background: silver; border: none; box-sizing: border-box; min-width: 80px; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-yfti-cnfc: 64;">
<span style="color: #333333; font-family: "segoe ui" , "sans-serif"; font-size: 11.5pt;">Used to generate a rowset from the input file or files.<o:p></o:p></span></div>
</td>
</tr>
<tr style="box-sizing: border-box; mso-yfti-irow: 1; mso-yfti-lastrow: yes;">
<td data-th="UDO" style="border-bottom: solid black 1.0pt; border: none; box-sizing: border-box; min-width: 80px; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-yfti-cnfc: 4;">
<b><span style="color: #333333; font-family: "segoe ui" , "sans-serif"; font-size: 11.5pt;">Outputters<o:p></o:p></span></b></div>
</td>
<td data-th="Description" style="border-bottom: solid black 1.0pt; border: none; box-sizing: border-box; min-width: 80px; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt;" valign="top"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="color: #333333; font-family: "segoe ui" , "sans-serif"; font-size: 11.5pt;">Used to transform a rowset into a file or set of files.<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt;">Extractors<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableLightShading" style="border-collapse: collapse; border: none; mso-border-bottom-alt: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-alt: solid black 1.0pt; mso-border-top-themecolor: text1; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: -1;">
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 167.4pt;" valign="top" width="223"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 5; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Extractors<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 311.4pt;" valign="top" width="415"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 1; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Descriptions<o:p></o:p></span></b></div>
</td>
</tr>
<tr style="mso-yfti-irow: 0;">
<td style="background: silver; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt; width: 167.4pt;" valign="top" width="223"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 68; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Extractors.Text()<o:p></o:p></span></b></div>
</td>
<td style="background: silver; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt; width: 311.4pt;" valign="top" width="415"><div class="MsoNormal" style="background: white; line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-yfti-cnfc: 64;">
<span style="font-family: "arial" , "sans-serif";">Provides extraction from delimited text files of
different encodings.<span style="color: black; mso-themecolor: text1; mso-themeshade: 191;"><o:p></o:p></span></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 1;">
<td style="border: none; padding: 0in 5.4pt 0in 5.4pt; width: 167.4pt;" valign="top" width="223"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 4; text-autospace: none;">
<b><span style="font-family: "arial" , "sans-serif";">Extractors.Csv()<o:p></o:p></span></b></div>
</td>
<td style="border: none; padding: 0in 5.4pt 0in 5.4pt; width: 311.4pt;" valign="top" width="415"><div class="MsoNormal" style="background: white; line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto;">
<span style="font-family: "arial" , "sans-serif";">Provides
extraction from comma-separated value (CSV) files of different encodings.<o:p></o:p></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 2; mso-yfti-lastrow: yes;">
<td style="background: silver; border-bottom: solid black 1.0pt; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 167.4pt;" valign="top" width="223"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 68; text-autospace: none;">
<b><span style="font-family: "arial" , "sans-serif";">Extractors.Tsv()<o:p></o:p></span></b></div>
</td>
<td style="background: silver; border-bottom: solid black 1.0pt; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 311.4pt;" valign="top" width="415"><div class="MsoNormal" style="background: white; line-height: normal; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-yfti-cnfc: 64;">
<span style="font-family: "arial" , "sans-serif";">Provides extraction from tab-separated value (TSV)
files of different encodings.<o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt;">Outputters<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<table border="1" cellpadding="0" cellspacing="0" class="MsoTableLightShading" style="border-collapse: collapse; border: none; mso-border-bottom-alt: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-alt: solid black 1.0pt; mso-border-top-themecolor: text1; mso-padding-alt: 0in 5.4pt 0in 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: -1;">
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 171.9pt;" valign="top" width="229"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 5; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Outputters<o:p></o:p></span></b></div>
</td>
<td style="border-bottom: solid black 1.0pt; border-left: none; border-right: none; border-top: solid black 1.0pt; mso-border-bottom-themecolor: text1; mso-border-top-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 306.9pt;" valign="top" width="409"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 1; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Description<o:p></o:p></span></b></div>
</td>
</tr>
<tr style="mso-yfti-irow: 0;">
<td style="background: silver; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt; width: 171.9pt;" valign="top" width="229"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 68; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Outputters.Text()<o:p></o:p></span></b></div>
</td>
<td style="background: silver; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; padding: 0in 5.4pt 0in 5.4pt; width: 306.9pt;" valign="top" width="409"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 64; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Provides outputting a rowset into a
variety of delimited text formats.<span style="color: black; mso-themecolor: text1; mso-themeshade: 191;"><o:p></o:p></span></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 1;">
<td style="border: none; padding: 0in 5.4pt 0in 5.4pt; width: 171.9pt;" valign="top" width="229"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 4; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Outputters.Csv()<o:p></o:p></span></b></div>
</td>
<td style="border: none; padding: 0in 5.4pt 0in 5.4pt; width: 306.9pt;" valign="top" width="409"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Provides outputting a rowset into a
comma-separated value (CSV) file of different encodings.<span style="color: black; mso-themecolor: text1; mso-themeshade: 191;"><o:p></o:p></span></span></div>
</td>
</tr>
<tr style="mso-yfti-irow: 2; mso-yfti-lastrow: yes;">
<td style="background: silver; border-bottom: solid black 1.0pt; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 171.9pt;" valign="top" width="229"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 68; text-autospace: none;">
<b><span style="color: black; font-family: "arial" , "sans-serif"; mso-themecolor: text1; mso-themeshade: 191;">Outputters.Tsv()<o:p></o:p></span></b></div>
</td>
<td style="background: silver; border-bottom: solid black 1.0pt; border: none; mso-background-themecolor: text1; mso-background-themetint: 63; mso-border-bottom-themecolor: text1; padding: 0in 5.4pt 0in 5.4pt; width: 306.9pt;" valign="top" width="409"><div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; mso-yfti-cnfc: 64; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Provides outputting a rowset into a
tab-separated value (TSV) file of different encodings.<span style="color: black; mso-themecolor: text1; mso-themeshade: 191;"><o:p></o:p></span></span></div>
</td>
</tr>
</tbody></table>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Hope you like it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "arial" , "sans-serif"; mso-themecolor: accent1;">Posted by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-22353539340137512852018-02-14T14:05:00.000+05:302018-02-14T14:14:49.197+05:30Azure Data Lake – U-SQL – Aggregation of Rowset<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;">In this article we are trying to discuss about basic
U-SQL SELECT query Aggregation of Rowsets Technique. Hope it will be
informative.<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbIDqp4NOf_LOd-fz6kO4DlLs4Iog6ksi_2Qm9jAsLxL11vOie2CZpVyq0jd2ssVi2yWN5pMmeb3Wbmm0rfmBHaMqpJtpl0srzHOg2zzqzDcT8XPmuExPXvYAchc_gv87iV3mgWb7DHC4/s1600/images.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="162" data-original-width="311" height="104" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbIDqp4NOf_LOd-fz6kO4DlLs4Iog6ksi_2Qm9jAsLxL11vOie2CZpVyq0jd2ssVi2yWN5pMmeb3Wbmm0rfmBHaMqpJtpl0srzHOg2zzqzDcT8XPmuExPXvYAchc_gv87iV3mgWb7DHC4/s200/images.jpg" width="200" /></a></div>
<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "arial" , "sans-serif"; font-size: 11.0pt; font-weight: normal;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt; line-height: 107%;">Aggregation of Rowset <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Here we are using example to understand Aggregation,
ORDER BY, GROUP BY and HAVING.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">The bellow query display the total duration of each
region using Aggregation SUM().<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/output/Searchlog-aggregation"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">+</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"_agg.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out2</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">+</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"_top5agg.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: "consolas"; font-size: 9.5pt;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: purple; font-family: "consolas"; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ORDER</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DESC</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Now we want to display Total duration of each reason and
with this we want top 5 durations in order.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/output/Searchlog-aggregation"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">+</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"_agg.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out2</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@outpref</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">+</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"_top5agg.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: "consolas"; font-size: 9.5pt;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: purple; font-family: "consolas"; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@res</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> *<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ORDER</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DESC</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FETCH</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #003200; font-family: "consolas"; font-size: 9.5pt;">5</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ROWS</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ORDER</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DESC</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@res</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@out2</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ORDER</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DESC</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Please note that, </span><span style="background: white; color: black; font-family: "segoe ui" , "sans-serif";">U-</span><span style="font-family: "arial" , "sans-serif";">SQL
Rowsets do not preserve their order for the next query. Thus, to order an
output, we need to add ORDER BY to the OUTPUT statement</span>.<o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt;">Having Clause<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Here we are using HAVING clause to restrict the output
effected by GROUP BY clause.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: "consolas"; font-size: 9.5pt;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@res</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: purple; font-family: "consolas"; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">) </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">AS</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">GROUP</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">HAVING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: purple; font-family: "consolas"; font-size: 9.5pt;">SUM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">) > </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #003200; font-family: "consolas"; font-size: 9.5pt;">200</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: "consolas"; font-size: 9.5pt;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">@res</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: "consolas"; font-size: 9.5pt;">"/output/Searchlog-having.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">ORDER</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">BY</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">TotalDuration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">DESC</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: "consolas"; font-size: 9.5pt;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: "consolas"; font-size: 9.5pt;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: "consolas"; font-size: 9.5pt;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "arial" , "sans-serif"; font-size: 14.0pt;">Built-in Function in U-SQL<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">We can find all the built-in Function in U-SQL<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";"><a href="https://msdn.microsoft.com/en-us/azure/data-lake-analytics/u-sql/built-in-functions-u-sql">https://msdn.microsoft.com/en-us/azure/data-lake-analytics/u-sql/built-in-functions-u-sql</a><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "arial" , "sans-serif";">Hope you like it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "arial" , "sans-serif"; mso-themecolor: accent1;">Posted by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com1tag:blogger.com,1999:blog-7468857462999845094.post-77096724820063866052018-02-10T20:27:00.000+05:302018-02-10T20:27:09.527+05:30Azure Data Lake – U-SQL – SELECT Transformation Rowsets<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss about basic
U-SQL SELECT query Transformation Rowsets Technique. Hope it will be
informative.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">What is U-SQL SELECT query Transformation Rowsets<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In my previous article we are just retrieving the
data from “SearchLog.tsv</span><span style="font-weight: normal;">” file to “SearchLog-scalar-variables.csv”.
It is just a simple file to file movement of data. </span><o:p></o:p></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Now we have to think some extra operation on
the Row set data before storing it into final destination. For example: some filtration,
Grouping, aggregate function in numerical values etc.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">When we are going to perform such kind of operation with
Row set before moving it to the destination, it is called SELECT Transformation
Rowsets.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Let’s Take a Simple Example to understand it<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">We are here taking the same example that we used previous
article to simply copy data from a file named “SearchLog.tsv</span>” and store
it into “SearchLog-scalar-variables.csv”.<o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">What we are doing in the Transformation part is, we are
just filtering the Row set region wise. We mean to say we are using a Boolean expression
in the WHERE clause of SELECT statement. Feeling quite comfortable ….<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">WHERE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> == </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"en-gb"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/output/SearchLog-transform-rowsets.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="color: black; font-family: Consolas; font-size: 9.5pt;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Here in U-SQL Part the transformation part is<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQipxIrqiXuwPMRqXaQUT9NykEjh5tTE7a92hD2NbaEu7RHLcLqeLStg152RTE8As-_qs5Yuvd6KMH3vD0RheSnL3kPwzbGANNohjWzez1XtPMKd1v1P2MeBsELIun2AoLfi7ZEXx3T2Q/s1600/fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="305" data-original-width="442" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQipxIrqiXuwPMRqXaQUT9NykEjh5tTE7a92hD2NbaEu7RHLcLqeLStg152RTE8As-_qs5Yuvd6KMH3vD0RheSnL3kPwzbGANNohjWzez1XtPMKd1v1P2MeBsELIun2AoLfi7ZEXx3T2Q/s400/fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">The WHERE clause contains a </span><span style="color: windowtext; text-decoration: none; text-underline: none;">C# Boolean expression</span><span style="-webkit-text-stroke-width: 0px; float: none; font-variant-caps: normal; font-variant-ligatures: normal; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">. We can use the C# expression language to do our own expressions
and functions. We can even perform more complex filtering by combining them
with logical conjunctions (ANDs) and disjunctions (ORs).</span><o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">We can use More than One Transformation Together<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">In the previous example, we are using a single
Transformation by using a WHERE clause with region.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">We can again use the previous filtered Row set for new Transformation.
We can do it multiple times. <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Please look at the bellow example. First we are using
Transformation to filtering Region. Now we are using DateTime.Parse() method
and a conjunction</span><span style="background: white; color: black; font-family: "Segoe UI","sans-serif";">.</span><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">WHERE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> == </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">"en-gb"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">SELECT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">, </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">WHERE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> >= </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">Parse</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">"2012/02/16"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">) <o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">AND</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;"> <= </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">Parse</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">(</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">"2012/02/17"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: yellow;">);<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@rs1</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/output/SearchLog-transform-datetime.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">The second query is operating on the result of the first
rowset, which creates a composite of the two filters. We can also reuse a
variable name, and the names are scoped lexically.</span><o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Hope you like it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-84391739770633331782018-02-07T14:53:00.002+05:302018-02-07T14:53:41.435+05:30Azure Data Lake – U-SQL Script Construction<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss about basic
U-SQL query construction Technique. Hope it will be informative.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">First U-SQL Project Solution Explorer<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="background: yellow; font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-highlight: yellow;">View </span><span style="background: yellow; font-family: Wingdings; font-size: 12.0pt; mso-ascii-font-family: Arial; mso-bidi-font-family: Arial; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-char-type: symbol; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-hansi-font-family: Arial; mso-highlight: yellow; mso-symbol-font-family: Wingdings;"><span style="mso-char-type: symbol; mso-symbol-font-family: Wingdings;">à</span></span><span style="background: yellow; font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-highlight: yellow;"> Solution Explorer</span><span style="font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"> <o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<br /></h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghiN6ASaZWSte6_eI149t0HwGaXMr7FFAdAePm1wnf0kxEc_qulhqy0S0jYlI2UEoEIUSMjsv86zQu9J0D8Q65fYfY-O5ibFz3Tc5SUC21cmLeDQ7cLd8jMR1s0q3s9f6rEsa7y7rndss/s1600/fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="202" data-original-width="249" height="324" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghiN6ASaZWSte6_eI149t0HwGaXMr7FFAdAePm1wnf0kxEc_qulhqy0S0jYlI2UEoEIUSMjsv86zQu9J0D8Q65fYfY-O5ibFz3Tc5SUC21cmLeDQ7cLd8jMR1s0q3s9f6rEsa7y7rndss/s400/fig-1.jpg" width="400" /></a></div>
<div>
<br /></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"><o:p> </o:p></span></h3>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">The solution contains single U-SQL project. Here in this
example it is “Test” project. Inside that project is the empty Script.usql file.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Notice that we can expand the .usql file node. It
contains a C# file. This is called the U-SQL<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Code-Behind file. U-SQL Code-behind is a convenience
feature of ADLToolsForVS and not a part of the U-SQL language itself.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Input and Output<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Compiling U-SQL file required Input. U-SQL script
transforms Input into Output. There are two types of Input and Output.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoListParagraphCxSpFirst" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 1.5in; margin-right: 0in; margin-top: 0in; mso-add-space: auto; mso-layout-grid-align: none; mso-list: l0 level1 lfo1; text-autospace: none; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Files<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 1.5in; margin-right: 0in; margin-top: 0in; mso-add-space: auto; mso-layout-grid-align: none; mso-list: l0 level1 lfo1; text-autospace: none; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">U-SQL
Table<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Location of Input and Output<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">During U-SQL Cloud Execution the inputs/outputs must all
be in the cloud - typically this means Azure Data Lake Store.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">During U-SQL Local Execution the inputs/outputs must all
be on your own box. There's a special name for this location: The U-SQL Local
Data Root.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">We can find the local data root by going to </span><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-highlight: yellow;">Tools
</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; font-family: Wingdings; font-size: 12.0pt; mso-ascii-font-family: Arial; mso-bidi-font-family: Arial; mso-bidi-font-size: 11.0pt; mso-char-type: symbol; mso-hansi-font-family: Arial; mso-highlight: yellow; mso-symbol-font-family: Wingdings;"><span style="mso-char-type: symbol; mso-symbol-font-family: Wingdings;">à</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-highlight: yellow;">
Data Lake </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; font-family: Wingdings; font-size: 12.0pt; mso-ascii-font-family: Arial; mso-bidi-font-family: Arial; mso-bidi-font-size: 11.0pt; mso-char-type: symbol; mso-hansi-font-family: Arial; mso-highlight: yellow; mso-symbol-font-family: Wingdings;"><span style="mso-char-type: symbol; mso-symbol-font-family: Wingdings;">à</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: yellow; font-family: "Arial","sans-serif"; font-size: 12.0pt; mso-bidi-font-size: 11.0pt; mso-highlight: yellow;">
Options and Settings</span></b><span style="font-family: "Arial","sans-serif";">.
It in the field called DataRoot at the top.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj93m5BnKq8cEOxEAqasCdAWdjqPVZMFNXsiWvw-EjfXbFO-XIBkboGppiWBjsUfsKq4SuncnqSxD6yB0bLrxncSJeLMOQfwcBvvHT1MATL-C9SwEZy1VjpaR_mMVsbkNcoVH22k-Huwi0/s1600/fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="646" data-original-width="1003" height="257" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj93m5BnKq8cEOxEAqasCdAWdjqPVZMFNXsiWvw-EjfXbFO-XIBkboGppiWBjsUfsKq4SuncnqSxD6yB0bLrxncSJeLMOQfwcBvvHT1MATL-C9SwEZy1VjpaR_mMVsbkNcoVH22k-Huwi0/s400/fig-2.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Sample U-SQL Script<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">This script don’t have any transformation steps.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/output/SearchLog-first-u-sql.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div style="background: white; margin-bottom: .0001pt; margin-bottom: 0in;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">This script doesn't have any
transformation steps. It reads from the source file called SearchLog.tsv,
schematizes it, and writes the rowset back into a file called
SearchLog-first-u-sql.csv.<o:p></o:p></span></div>
<div style="-webkit-text-stroke-width: 0px; background: white; font-variant-caps: normal; font-variant-ligatures: normal; margin-bottom: .0001pt; margin-bottom: 0in; margin-top: 1rem; orphans: 2; text-align: start; text-decoration-color: initial; text-decoration-style: initial; widows: 2; word-spacing: 0px;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">Notice the question mark next to the data type in the Duration field.
It means that the Duration field could be null.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Here we have to understand Rowset, Extract and Output.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmT1oacsC8QlPBqQGhwOeJjB_MyU19CcPMyD9bCeOTXh2GoirG54XXHU3MESdIN4bpvGHzu-wy_0DLVrSMhZpjdzUF33pMG9nJzOTPB-qOlZy3GH5takop3bJJYUuIirtvAq_xZ3cOnZE/s1600/fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="361" data-original-width="623" height="231" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmT1oacsC8QlPBqQGhwOeJjB_MyU19CcPMyD9bCeOTXh2GoirG54XXHU3MESdIN4bpvGHzu-wy_0DLVrSMhZpjdzUF33pMG9nJzOTPB-qOlZy3GH5takop3bJJYUuIirtvAq_xZ3cOnZE/s400/fig-3.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-left: 28.5pt; mso-list: l1 level1 lfo2; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b style="mso-bidi-font-weight: normal;"><u><span style="font-family: "Arial","sans-serif";">Rowset variables:</span></u></b><span style="font-family: "Arial","sans-serif";"> Each query expression that produces a
rowset can be assigned to a variable. U-SQL follows the T-SQL variable naming
pattern (@searchlog, for example) in the script.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-left: 28.5pt; mso-list: l1 level1 lfo2; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">The <b style="mso-bidi-font-weight: normal;"><u>EXTRACT</u></b> keyword reads data
from a file and defines the schema on read. Extractors.Tsv is a
built-in U-SQL extractor for tab-separated-value files. We can develop custom
extractors.<br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--><o:p></o:p></span></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-left: 28.5pt; mso-list: l1 level1 lfo2; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; tab-stops: list .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 11.0pt; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">The <b style="mso-bidi-font-weight: normal;"><u>OUTPUT</u></b> writes data from a
rowset to a file. Outputters.Csv() is a built-in U-SQL outputter to
create a comma-separated-value file. You can develop custom outputters.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">The EXTRACT and OUTPUT statements use file paths. File
paths can be absolute or relative</span>.<o:p></o:p></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Here we can use scalar variable to make the script maintenance
easier.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@in</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/Samples/Data/SearchLog.tsv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DECLARE</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@out</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> = </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #a05000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">"/output/SearchLog-scalar-variables.csv"</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> =<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">EXTRACT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">UserId</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Start</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #006060; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">DateTime</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Region</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Query</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Duration</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">int</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">?,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Urls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">,<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">ClickedUrls</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">string</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">FROM</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@in</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Extractors</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Tsv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">OUTPUT</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@searchlog</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;"> </span><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">TO</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">@out</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"><span style="mso-spacerun: yes;">
</span></span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #0000d0; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">USING</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;"> </span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Outputters</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">.</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: #505000; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">Csv</span></b><b style="mso-bidi-font-weight: normal;"><span style="background: white; color: black; font-family: Consolas; font-size: 9.5pt; mso-highlight: white;">();</span></b><b style="mso-bidi-font-weight: normal;"><span style="font-family: "Arial","sans-serif";"><o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: "Arial","sans-serif";">Hope you like it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b style="mso-bidi-font-weight: normal;"><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<br /></div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com4tag:blogger.com,1999:blog-7468857462999845094.post-50031116677839532952018-02-04T22:22:00.002+05:302018-02-04T22:22:48.149+05:30Azure Data Lake – Running U-SQL Script<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">In this article we are trying to discuss about
U-SQL query and how you can run it. Regarding Data Lake introduction, you can
refer my previous article named “<a href="http://sqlknowledgebank.blogspot.in/2018/01/azure-data-lake-introductory.html"><b><span style="color: windowtext; font-family: "Times New Roman", serif; text-decoration-line: none;">Azure Data Lake Introductory</span></b></a>”.<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"> </span></h3>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Types of U-SQL Run<o:p></o:p></span></b></div>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">There are two type of U-SQL run<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l1 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">U-SQL Local Execution<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: .5in; margin-right: 0in; margin-top: 9.0pt; mso-list: l1 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Arial;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">U-SQL Cloud Execution<o:p></o:p></span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;"> </span></h3>
<h3 style="background: white; margin-bottom: .0001pt; margin-bottom: 0in; margin-left: 0in; margin-right: 0in; margin-top: 9.0pt;">
<u><span style="font-family: "Arial","sans-serif"; font-size: 11.0pt; mso-bidi-font-weight: normal; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin;">U-SQL Local Execution<o:p></o:p></span></u></h3>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We can run
the U-SQL from our local machine. Here we are not using Azure resources. So
there is no additional cost for that. I personally preferred it is a best
method to learn U-SQL.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">U-SQL Cloud Execution<o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">We can run U-SQL
scripts in Azure in the context of a Data Lake Analytics account.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">The data read or
written by the script will also be in Azure - typically in an Azure Data<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: ArialMT; font-size: 11.5pt; line-height: 107%; mso-bidi-font-family: ArialMT;">Lake Store account.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Preparing your Local machine for running U-SQL<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Visual studio
needed<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Visual
Studio 2013, 2015 <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo2; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Visual
Studio 2017 (All Integrated)<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">Azure Data Lake Tools for Visual Studio (ADLToolsForVS)<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;"><br />
</span></b><span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">If we have Visual Studio 2013 or Visual Studio 2015, we need to run
the “Azure Data Lake Tools for Visual Studio (ADLToolsForVS)”<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">Here is the link:<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"><a href="https://www.microsoft.com/en-us/download/details.aspx?id=49504">https://www.microsoft.com/en-us/download/details.aspx?id=49504</a><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkbcARa_X1GqQcoUupa2PcEOdj8doyFSJrDuqiHiCKNB6ii2LS56l258fEsf5XyYUCmJiFRnVSkL0mjfNkOiTd2HMitM9OybYiQ0vDMQFMls41zvaisCv6EA6x6WJOkkQ_x54CcPad-Cg/s1600/fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="592" data-original-width="1224" height="192" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkbcARa_X1GqQcoUupa2PcEOdj8doyFSJrDuqiHiCKNB6ii2LS56l258fEsf5XyYUCmJiFRnVSkL0mjfNkOiTd2HMitM9OybYiQ0vDMQFMls41zvaisCv6EA6x6WJOkkQ_x54CcPad-Cg/s400/fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv88jqnXcOaYfyGzRXXuoZaB1Xe4-dj26SZzmlVC_iTk0FtL-RYrZkxqp5ywCDlPxntPzxnXplJnyX-UQhtEIG_8ve9cyWi8ptpFK8tmpgnivI5g4hKKJHgr9Kn-DrrymNRTSTmGYfH5Y/s1600/fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="226" data-original-width="773" height="116" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiv88jqnXcOaYfyGzRXXuoZaB1Xe4-dj26SZzmlVC_iTk0FtL-RYrZkxqp5ywCDlPxntPzxnXplJnyX-UQhtEIG_8ve9cyWi8ptpFK8tmpgnivI5g4hKKJHgr9Kn-DrrymNRTSTmGYfH5Y/s400/fig-2.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">How verify that ADLToolForVS is installed<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">We have to verify the
installation of ADLToolForVS. For that open the visual studio and choose the <b>Tool</b> menu. If we find any Data Lake
then it means that ADLToolVS is installed successfully.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1_Vc8QPJ6fZ-yyJJ290O4xOmagCvABoXMtCOeBkSFD_T6yOMQpC7Oh3bNW3YOnrQipepgjbGgJs1tBJc3scrwFeoCNiEEzw75qbZU_sBVAk4YOhuwAbiMef2VJdFaqeo0xPCgYeRDt2k/s1600/fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="423" data-original-width="614" height="275" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1_Vc8QPJ6fZ-yyJJ290O4xOmagCvABoXMtCOeBkSFD_T6yOMQpC7Oh3bNW3YOnrQipepgjbGgJs1tBJc3scrwFeoCNiEEzw75qbZU_sBVAk4YOhuwAbiMef2VJdFaqeo0xPCgYeRDt2k/s400/fig-3.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">How to open U-SQL Editor<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">To open U-SQL Editor
in Visual Studio<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"><br />
File </span><span style="font-family: Wingdings; font-size: 11.5pt; mso-ascii-font-family: ArialMT; mso-bidi-font-family: ArialMT; mso-char-type: symbol; mso-hansi-font-family: ArialMT; mso-symbol-font-family: Wingdings;">à</span><span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"> New </span><span style="font-family: Wingdings; font-size: 11.5pt; mso-ascii-font-family: ArialMT; mso-bidi-font-family: ArialMT; mso-char-type: symbol; mso-hansi-font-family: ArialMT; mso-symbol-font-family: Wingdings;">à</span><span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;"> Project<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCRyeRlcN50t2uLsOdEShN2I7J3cSdog7ETw4d7ywrUhpqqMUMJqKHFxYeU9zN7lme44skxUA1IMuwZPMrOgBAYPt5IJKmuh7dCanvMg_xYcFyT_YH-J3VbAYip4ql6cUjAptsv9lD5WQ/s1600/fig-4.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="582" data-original-width="955" height="243" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgCRyeRlcN50t2uLsOdEShN2I7J3cSdog7ETw4d7ywrUhpqqMUMJqKHFxYeU9zN7lme44skxUA1IMuwZPMrOgBAYPt5IJKmuh7dCanvMg_xYcFyT_YH-J3VbAYip4ql6cUjAptsv9lD5WQ/s400/fig-4.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; mso-bidi-font-size: 11.0pt;">The Script File<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">At this point you
should notice that an empty U-SQL script has been created called<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">"Script.usql".<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh56jDHO6WCAjMCXPbxeLWSDAaUT7PKtqJIX7Gkh-bZXI8HYDIjg9dtOTIzRgDTvdl78iMJ8aWZ3KNjDVl1RWC-HCzSv2P9XoM4zOGmmXI0MJ83Y2ngDAEXnre6_ODG5LVM6ts-dANGUjc/s1600/fig-5.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="150" data-original-width="968" height="61" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh56jDHO6WCAjMCXPbxeLWSDAaUT7PKtqJIX7Gkh-bZXI8HYDIjg9dtOTIzRgDTvdl78iMJ8aWZ3KNjDVl1RWC-HCzSv2P9XoM4zOGmmXI0MJ83Y2ngDAEXnre6_ODG5LVM6ts-dANGUjc/s400/fig-5.jpg" width="400" /></a></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">The Submit button will
run the script. The dropdown next to the Submit button says<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">(Local). This means
that Submit will run the script in Local Execution mode. <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT;">Hope you like it.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<br />
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: #5b9bd5; font-family: ArialMT; font-size: 11.5pt; mso-bidi-font-family: ArialMT; mso-themecolor: accent1;">Posted By: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com2tag:blogger.com,1999:blog-7468857462999845094.post-40097126021165221842018-01-13T10:52:00.000+05:302018-01-13T10:52:08.572+05:30Azure Data Lake Introductory<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction</span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">This is my
first article related to Azure. This article is dedicated to my father Late
Subal Chandra Das, who always inspire me to do something new. I missed you lot
Baba.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihxOyBYFOYBR19snVXlYnTCIH7MQwlbTlIPitg7fUYV7Uq7GvzD8MEr0H6cMYL6NVkHRDvxN_zAY6GsNfvSnmFVwW5sr7INY4n-7WaQTkcZPTO-SkypH1VJT_iYbkPArB5JEqjc-YgYp0/s1600/logo.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="164" data-original-width="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihxOyBYFOYBR19snVXlYnTCIH7MQwlbTlIPitg7fUYV7Uq7GvzD8MEr0H6cMYL6NVkHRDvxN_zAY6GsNfvSnmFVwW5sr7INY4n-7WaQTkcZPTO-SkypH1VJT_iYbkPArB5JEqjc-YgYp0/s1600/logo.jpg" /></a></div>
<br />
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">This article
is related to the general architecture of Azure Data Lake. Hope it will be a
good foundation to start with Azure Data Lake. The article is a representation
of my understanding with Azure Data Lake. <o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">In coming
days we are going to be more advanced with it. Hope it will be informative.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">What is Data Lake<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"> </span><span style="font-family: Arial, sans-serif;">Before jump
into Azure Data Lake, we have to understand the concept behind Data Lake.</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">A data lake
is a storage repository that holds a vast amount of raw data in its native
format until it is needed. While a hierarchical data warehouse stores data in
files or folders, a data lake uses a flat architecture to store data. Each data
element in a lake is assigned a unique identifier and tagged with a set of
extended metadata tags. When a business question arises, the data lake can be
queried for relevant data, and that smaller set of data can then be analyzed to
help answer the question.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">A data lake,
on the other hand, maintains data in their native formats and handles the three
Vs of big data (Volume, Velocity and Variety) while providing tools for
analysis, querying, and processing. Data Lake eliminates all the restrictions
of a typical data warehouse system by providing unlimited space, unrestricted
file size, schema on read, and various ways to access data (including
programming, SQL-like queries, and REST calls).<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6eNZmw1VrQ-Fnj2GX-lRz11KsG4cjA6ng7Av_vPZz33-OVQwlsWPSORMJHJWRfBovuQDYtGbkLiIa5i2AY0j_WoOWrm5rUjq1DuRFNJZGk06wip1jaK8VlW0moOirPGCfTKbw6MmTUao/s1600/fig-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="920" data-original-width="1600" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6eNZmw1VrQ-Fnj2GX-lRz11KsG4cjA6ng7Av_vPZz33-OVQwlsWPSORMJHJWRfBovuQDYtGbkLiIa5i2AY0j_WoOWrm5rUjq1DuRFNJZGk06wip1jaK8VlW0moOirPGCfTKbw6MmTUao/s400/fig-1.png" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">With the
emergence of Hadoop (including HDFS and YARN), the benefits of data lake –
previously available only to the most resource-rich companies like Google,
Yahoo, and Facebook – became a practical reality for just about anyone. Now,
organizations who had been generating and gathering data on a large scale but
had struggled to store and process them in a meaningful way, have more options.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Feature of Azure Data Lake<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Azure Data
Lake is a new kind of data lake bock from Microsoft Azure. The features that it
offers are mentioned below.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• The ability to store and analyze data
of any kind and size.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• Multiple access methods including
U-SQL, Spark, Hive, HBase, and Storm.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• Built on YARN and HDFS.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• Dynamic scaling to match your
business priorities.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• Enterprise-grade security with Azure
Active Directory.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">• Managed and supported with an
enterprise-grade SLA.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Parts of Azure Data Lake<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Broadly the
Azure Data Lake is classified into three parts<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF8YW9_gRJNZgGlJGLckB0_CU0KZBTDwLm9DybBmjQ0jWyoPd4d1SUghTeDd04LvvxFJwTD7eG2363LWfJsuHAqskoDEt74m2L3yM-UbVVKFh1_wB0L_kBvbHBqp3hy26hBx6Yt8Pw3ak/s1600/fig-2.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="355" data-original-width="521" height="272" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF8YW9_gRJNZgGlJGLckB0_CU0KZBTDwLm9DybBmjQ0jWyoPd4d1SUghTeDd04LvvxFJwTD7eG2363LWfJsuHAqskoDEt74m2L3yM-UbVVKFh1_wB0L_kBvbHBqp3hy26hBx6Yt8Pw3ak/s400/fig-2.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Azure Data Lake Store<o:p></o:p></span></u></b></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDFe6hJQaJB7jGUuNOnGy7eap0Fwpht4hSqWtDc71PtrMauhppncyUvvwS94wWt9lGR-v8AzZy_BEMVyL-FOpdp3LQ3CziocD_HL9tUog4Nid5_WCzhPkySnMcJq4R1yD-cQbMXLRF8sA/s1600/fig-3.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="321" data-original-width="513" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDFe6hJQaJB7jGUuNOnGy7eap0Fwpht4hSqWtDc71PtrMauhppncyUvvwS94wWt9lGR-v8AzZy_BEMVyL-FOpdp3LQ3CziocD_HL9tUog4Nid5_WCzhPkySnMcJq4R1yD-cQbMXLRF8sA/s400/fig-3.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">The Data Lake
store provides a single repository where organizations upload data of just
about infinite volume. The store is designed for high-performance processing
and analytics from HDFS applications and tools, including support for low
latency workloads. In the store, data can be shared for collaboration with
enterprise-grade security.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Azure Data Lake analytics<o:p></o:p></span></u></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Data Lake
analytics is a distributed analytics service built on Apache YARN that
compliments the Data Lake store. The analytics service can handle jobs of any
scale instantly with on-demand processing power and a pay-as-you-go model
that’s very cost effective for short term or on-demand jobs. It includes a
scalable distributed runtime called U-SQL, a language that unifies the benefits
of SQL with the expressive power of user code.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Azure HDInsight <o:p></o:p></span></u></b></div>
<div class="MsoNormal">
<span style="font-family: Arial, sans-serif;">Azure
HDInsight is a full stack Hadoop Platform as a Service from Azure. Built on top
of Hortonworks Data Platform (HDP), it provides Apache Hadoop, Spark, HBase,
and Storm clusters.</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">References:<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><a href="https://azure.microsoft.com/en-in/solutions/data-lake/">https://azure.microsoft.com/en-in/solutions/data-lake/</a><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope you like
it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<br /></div>
</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com20tag:blogger.com,1999:blog-7468857462999845094.post-26219806116358258212017-12-29T13:40:00.001+05:302017-12-29T13:40:21.616+05:30SSIS Data Flow Error Handling with Description and Column Name<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Introduction<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Finding the
Error Description is a simple one line code but finding Error Column name is
not so much simple. Microsoft made it easy from SQL 2016, when modified API comes.
But before 2016 it’s really a difficult job to find the Error Column Name by
using Script Task and Script Component. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">This article
is trying to provide a simple step by step solution to find the Error
Description and Error column name.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Scenario<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">We are trying
to insert records from Flat file to our SQL Table. The flat file has two
columns named “roll” and “name”. Sometimes name contains huge text and don’t support
by our Table object (Size of the name column in table object is Varchar(50))
and hence generate Error. We are trying to display Error description and the
column name that generate the Error.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Control/Data Flow Details<o:p></o:p></span></b></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;"><br /></span></b></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3eE6edPdw8ZJDHQEudVX9suwfzeRp72Qyaeu8PnQYgYbXBPc_nG4ZMsjD3wW30HhM8VuhCLYqUFpO4I1YVU_bz8oiWB4Qtnbmu4kjS3ru04qv8kuFereJVf_xEg8QvBSXxru0CfgL_X0/s1600/Fig-1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="297" data-original-width="607" height="195" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3eE6edPdw8ZJDHQEudVX9suwfzeRp72Qyaeu8PnQYgYbXBPc_nG4ZMsjD3wW30HhM8VuhCLYqUFpO4I1YVU_bz8oiWB4Qtnbmu4kjS3ru04qv8kuFereJVf_xEg8QvBSXxru0CfgL_X0/s400/Fig-1.jpg" width="400" /></a></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Control Flow – Get Linage ID Task<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">This Script
Task is used to grade the Linage id at run time and save it in a text file.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">What to do:<o:p></o:p></span></u></b></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Create
a text file C:\temp\write.txt<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Create
two package level variable named <b>User::execsObj</b>
and <b>User::lineageIds</b> as Object type
and pass them as Read Write variable of script task.<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">In
Script Task add assembly named <b>Microsoft.SqlServer.DTSPipelineWrap</b><o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<span style="font-family: "Arial","sans-serif";"><b><br /></b></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Namespace needed<o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> System;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> System.Data;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Runtime;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Pipeline.Wrapper;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> System.Windows.Forms;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> System.Collections.Generic;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span></b><b><span style="font-family: Consolas; font-size: 9.5pt;"> System.IO;<o:p></o:p></span></b></div>
<div class="MsoNormal">
<br /></div>
<br />
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Script Task Code in C#</span></u></b></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: Consolas; font-size: 9.5pt;">public</span><span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">void</span> Main()<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: green;">// Grab the executables so we have to something to iterate
over, and initialize our lineageIDs list</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: green;">// Why the executables?
Well, SSIS won't let us store a reference to the Package itself...</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">Dts.Variables[<span style="color: #a31515;">"User::execsObj"</span>].Value
=(<span style="color: #2b91af;">Package</span>)Dts.Variables[<span style="color: #a31515;">"User::execsObj"</span>].Parent).Executables;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">Dts.Variables[<span style="color: #a31515;">"User::lineageIds"</span>].Value
= <span style="color: blue;">new</span> <span style="color: #2b91af;">Dictionary</span><<span style="color: blue;">int</span>, <span style="color: blue;">string</span>>();<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">lineageIds = (<span style="color: #2b91af;">Dictionary</span><<span style="color: blue;">int</span>, <span style="color: blue;">string</span>>)Dts.Variables[<span style="color: #a31515;">"User::lineageIds"</span>].Value;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: #2b91af;">Executables</span> execs = (<span style="color: #2b91af;">Executables</span>)Dts.Variables[<span style="color: #a31515;">"User::execsObj"</span>].Value;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">ReadExecutables(execs);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: green;">// Just proof of concept to see the results before you
dedicate your time to the solution</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: green;">// Delete this code in your actual implementation</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;">using</span> (<span style="color: #2b91af;">StreamWriter</span>
writetext = <span style="color: blue;">new</span> <span style="color: #2b91af;">StreamWriter</span>(<span style="color: #a31515;">@"C:\temp\write.txt"</span>, <span style="color: blue;">true</span>))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
<span style="color: blue;">foreach</span> (<span style="color: blue;">var</span>
kvp <span style="color: blue;">in</span> lineageIds)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> writetext.WriteLine(kvp.Key + <span style="color: #a31515;">" :
"</span> + kvp.Value);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
Dts.TaskResult = (<span style="color: blue;">int</span>)<span style="color: #2b91af;">ScriptResults</span>.Success;</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">}</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;"><br /></span></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;">private</span> <span style="color: blue;">void</span>
ReadExecutables(<span style="color: #2b91af;">Executables</span> executables)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">Executable</span>
pkgExecutable <span style="color: blue;">in</span> executables)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">if</span> (<span style="color: blue;">object</span>.ReferenceEquals(pkgExecutable.GetType(), </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;"> typeof</span>(Microsoft.SqlServer.Dts.Runtime.<span style="color: #2b91af;">TaskHost</span>)))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
<span style="color: #2b91af;">TaskHost</span> pkgExecTaskHost = (<span style="color: #2b91af;">TaskHost</span>)pkgExecutable;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">if</span>
(pkgExecTaskHost.CreationName.StartsWith(<span style="color: #a31515;">"SSIS.Pipeline"</span>))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
ProcessDataFlowTask(pkgExecTaskHost);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">else</span> <span style="color: blue;">if</span>
(<span style="color: blue;">object</span>.ReferenceEquals(pkgExecutable.GetType(), </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;"> typeof</span>(Microsoft.SqlServer.Dts.Runtime.<span style="color: #2b91af;">ForEachLoop</span>)))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
{<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
<span style="color: green;">// Recurse into FELCs</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
ReadExecutables(((<span style="color: #2b91af;">ForEachLoop</span>)pkgExecutable).Executables);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><span style="color: blue;">private</span> <span style="color: blue;">void</span>
ProcessDataFlowTask(<span style="color: #2b91af;">TaskHost</span>
currentDataFlowTask)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">{</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> </span><span style="color: #2b91af; font-family: Consolas; font-size: 9.5pt;">MainPipe</span><span style="font-family: Consolas; font-size: 9.5pt;"> currentDataFlow = (</span><span style="color: #2b91af; font-family: Consolas; font-size: 9.5pt;">MainPipe</span><span style="font-family: Consolas; font-size: 9.5pt;">)currentDataFlowTask.InnerObject;</span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"><br /></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">IDTSComponentMetaData100</span>
currentComponent <span style="color: blue;">in</span> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> currentDataFlow.ComponentMetaDataCollection)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: green;">// Get the inputs in the component.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">IDTSInput100</span>
currentInput <span style="color: blue;">in</span>
currentComponent.InputCollection)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">IDTSInputColumn100</span>
currentInputColumn <span style="color: blue;">in</span> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> currentInput.InputColumnCollection)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
lineageIds.Add(currentInputColumn.ID, currentInputColumn.Name);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: green;">// Get the outputs in the component.</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">foreach</span> (<span style="color: #2b91af;">IDTSOutput100</span>
currentOutput <span style="color: blue;">in</span>
currentComponent.OutputCollection)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
<span style="color: blue;">foreach</span> (<span style="color: #2b91af;">IDTSOutputColumn100</span>
currentoutputColumn <span style="color: blue;">in</span> </span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> currentOutput.OutputColumnCollection)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
lineageIds.Add(currentoutputColumn.ID, currentoutputColumn.Name);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">}<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: "Arial","sans-serif"; font-size: 14.0pt; line-height: 107%; mso-bidi-font-size: 11.0pt;">Adding Script Component in Data Flow Task<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Now add
script component in data flow task.<o:p></o:p></span></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";"><br /></span></u></b></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">What to do:<o:p></o:p></span></u></b></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Just
pass the variable named <b>User::lineageIds
</b>as Read variable<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Script
Component input columns as ErrorCode and ErrorColumn<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "Arial","sans-serif"; mso-fareast-font-family: Arial;">3.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span style="font-family: "Arial","sans-serif";">Script
Component output column as ErrorDescription and ErrorColumnDescription with
DT_STR data type.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Name space<o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span><span style="font-family: Consolas; font-size: 9.5pt;"> System;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span><span style="font-family: Consolas; font-size: 9.5pt;"> System.Data;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span><span style="font-family: Consolas; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Pipeline.Wrapper;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<b><span style="color: blue; font-family: Consolas; font-size: 9.5pt;">using</span><span style="font-family: Consolas; font-size: 9.5pt;"> Microsoft.SqlServer.Dts.Runtime.Wrapper;<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: Consolas; font-size: 9.5pt;"><b>using</b></span><span style="font-family: Consolas; font-size: 9.5pt;"><b> System.Collections.Generic;</b><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><u><span style="font-family: "Arial","sans-serif";">Script Component Code in C# <o:p></o:p></span></u></b></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="color: blue; font-family: Consolas; font-size: 9.5pt;">public</span><span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">override</span> <span style="color: blue;">void</span> Input0_ProcessInputRow(<span style="color: #2b91af;">Input0Buffer</span>
Row)<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: #2b91af;">Dictionary</span><<span style="color: blue;">int</span>,
<span style="color: blue;">string</span>> lineageIds = (<span style="color: #2b91af;">Dictionary</span><<span style="color: blue;">int</span>,
<span style="color: blue;">string</span>>)Variables.lineageIds;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">int</span>? colNum = Row.ErrorColumn;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">if</span> (colNum.HasValue && (lineageIds != <span style="color: blue;">null</span>))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> {<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">if</span> (lineageIds.ContainsKey(colNum.Value))<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
Row.ErrorColumnDescription = lineageIds[colNum.Value];<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<br /></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> <span style="color: blue;">else</span><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
Row.ErrorColumnDescription = <span style="color: #a31515;">"Row
error"</span>;<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;">
Row.ErrorDescription = <span style="color: blue;">this</span>.ComponentMetaData.GetErrorDescription(Row.ErrorCode);<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0in; mso-layout-grid-align: none; text-autospace: none;">
<span style="font-family: Consolas; font-size: 9.5pt;"> }<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "Arial","sans-serif";">Hope you find
it informative.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="color: #5b9bd5; font-family: "Arial","sans-serif"; mso-themecolor: accent1;">Posted
by: MR. JOYDEEP DAS<o:p></o:p></span></b></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<br /></div>
</div>
MR. JOYDEEP DAS (MVP-SQL Server Year 2012, MVB - D-Zone, MCDBA, MCSE, ADSE, CSI)http://www.blogger.com/profile/15813518881162145877noreply@blogger.com54