Introduction and Resources

Welcome to my 7-post series on Complex Event Processing with Mircosoft StreamInsight. Enjoy!

Microsoft StreamInsight is a very lightweight, powerful and for .NET developers easy to use Complex Event Processing (CEP) engine. It uses .NET LINQ queries to write queries on data streams.

With minimal effort, StreamInsight allows you to answer questions such as “what is the one minute aggregate of my value X grouped by dimension D”. This series of blog posts explains the concepts behind CEP and StreamInsight and implements a sample application that can be used to analyze live data from Twitter.

Index of this blog series:

Part 1 (03.10.13): CEP with StreamInsight and Twitter 1:
Introduction and Resources

Part 2 (06.10.13): CEP with StreamInsight and Twitter 2:
What are CEP and StreamInsight

Part 3 (13.10.13): CEP with StreamInsight and Twitter 3:
StreamInsight Programming

Part 4 (20.10.13): CEP with StreamInsight and Twitter 4:
Reactive Programming

Part 5 (27.10.13): CEP with StreamInsight and Twitter 5:
The Sample Application 1

Part 6 (03.11.13): CEP with StreamInsight and Twitter 6:
The Sample Application 2

Part 7 (10.11.13): CEP with StreamInsight and Twitter 7:
The Sample Application 3

The best resources that I found:

Video: “What’s new in Complex Event Processing using StreamInsight for SqlServer 2.1” by Torsten Grabs, Principal Program Manager Lead, Microsoft

http://www.youtube.com/watch?v=2k1AO5Wr_kk

Video: “My Campus Walk: I walk, StreamInsight analyzes, Worldwide Telescope visualizes.” Interesting project about visualization of a walk path from geo-spatial data processed by StreamInsight.

http://www.youtube.com/watch?v=Zi7JJmC4Nxw

Video: “Heat mapping H5N1 Outbreaks with StreamInsight and Binq Maps”. Really awesome demo application that feeds processed stream data to a Bing Maps application.

http://www.youtube.com/watch?v=cctmJZZDBlg

Tools

A tool I absolutely recommend, especially for StreamInsight Development,  is LINQPad (http://www.linqpad.net/). LINQPad includes the following StreamInsight sample packages that can be executed and played with:

• 101 StreamInsight Samples
• Hitchhikers Guide to StreamInsight
• Performance Counter Samples
• Reactive Extensions
• LINQ in Action

Important Note:
If you use the free version of LINQPad you will not get any intellisense or auto-complete functionality. In my opinion this makes it useless. As soon as I bought a personal license (about USD30), LINQPad turned into one of my favorite dev tools of all times. It is extremely lightweight and understands C#, VB, F#, SQL, LINQ, ADO.NET, StreamInsight, etc.

Samples

TwitterStream Sample: The application that we are going to develop in the following blog posts:

Hitchhikers Guide to StreamInsight Queries: Very comprehensive guide to writing queries in StreamInsight. Code is included in the codeplex download below. Samples are included in LINQPad.
http://blogs.msdn.com/b/streaminsight/archive/2012/08/01/a-hitchhiker-s-guide-to-streaminsight-2-1-queries.aspx

StreamInsight Product Team Samples 2.1
http://streaminsight.codeplex.com/releases/view/90143

StreamInsight Product Team Samples 2.0
Very important collection of samples such as: TextFileReaderAdapter, TextFileWriterAdapter, SqlAdapter, WcfAdapter and lots of applications.
http://streaminsight.codeplex.com/releases/view/46435

JAhlen StreamInsight Example Adapters v2.1
Stock trading sample application.
http://streaminsight.codeplex.com/releases/view/97583

Blogs/MSDN/Technet/Forums:

Blog: Official Blog of the Microsoft StreamInsight Dev Team
http://blogs.msdn.com/b/streaminsight/

Blog: StreamInsight explained to non-programmers
http://social.technet.microsoft.com/wiki/contents/articles/14437.streaminsight-for-non-programmers.aspx

Blog: Getting Started with StreamInsight 2.1
http://blogs.msdn.com/b/streaminsight/archive/2012/09/18/getting-started-with-streaminsight-2-1.aspx

Technet: Microsoft StreamInsight
http://technet.microsoft.com/en-us/library/hh750618(v=sql.10).aspx

MSDN: Microsoft StreamInsight
http://msdn.microsoft.com/en-us/library/hh750618(v=sql.10).aspx

MSDN Forum: SQL Server –> Microsoft StreamInsight
http://social.msdn.microsoft.com/Forums/sqlserver/en-US/home?forum=streaminsight

Next Post:

Using Complex Event Processing (CEP) with Microsoft StreamInsight to Analyze Twitter Tweets 2: What are CEP and StreamInsight?