SQL http://jasonfollas.com/blog/category/6.aspx SQL en-US Jason Follas jason@jasonfollas.com Subtext Version 2.0.0.43 Spatial Data and the Entity Framework http://jasonfollas.com/blog/archive/2010/02/14/spatial-data-and-the-entity-framework.aspx <p><span style="font-style: italic;">July 28, 2011 </span><span style="font-style: italic;">Note: </span><span style="font-style: italic;">This is an outdated post. Recently, the ADO.NET team has released a CTP with Spatial support as a first class citizen of the Entity Framework!!!</span> <span style="font-style: italic;"><span style="font-style: italic;">See the following posts that I wrote as I explored the new API:<br /> </span></span></p> <p><a href="http://www.jasonfollas.com/blog/archive/2011/07/20/entity-framework-spatial-first-look.aspx"><span style="font-style: italic;"><span style="font-style: italic;">http://www.jasonfollas.com/blog/archive/2011/07/20/entity-framework-spatial-first-look.aspx</span></span></a></p> <p><a href="http://www.jasonfollas.com/blog/archive/2011/07/21/entity-framework-spatial-dbgeography-members.aspx" target="_blank"><span style="font-style: italic;"><span style="font-style: italic;">http://www.jasonfollas.com/blog/archive/2011/07/21/entity-framework-spatial-dbgeography-members.aspx</span></span></a></p> <p><a href="http://www.jasonfollas.com/blog/archive/2011/07/27/entity-framework-spatial-a-real-world-example.aspx" target="_blank"><span style="font-style: italic;"><span style="font-style: italic;">http://www.jasonfollas.com/blog/archive/2011/07/27/entity-framework-spatial-a-real-world-example.aspx</span></span></a></p> <hr style="width: 100%; height: 2px;" /> The Entity Framework does not support using User Defined Types (at least in the SQLCLR sense of the term) as properties of an entity. Yesterday, <a target="_blank" href="http://thedatafarm.com/blog/">Julie Lerman</a> contacted me to see if we could find a workaround to this current limitation, particularly for the SQL Server Spatial Types (Geometry and Geography). <p>Whenever I hear of someone wanting to use Spatial data in their application, my first thought is always “what do they want to do with the data once they have it?” This is because most of the time (in my limited observation), an application does not need the spatial data itself, but rather, it just needs to use that data in the predicate of a query (i.e., the query results contain no spatial information). For example, an application might want all zipcodes that are within 50 km of a point, but the application doesn’t need the actual shapes that define each zip code.</p> <p>But, assuming that the developer knows what they are doing and has a legitimate reason to include a spatial type in the results, then how can they use the Entity Framework to get the spatial data into their application? That was our quest.</p> <h3>Entity Framework Primitive Types</h3> <p>Admittedly, I know very little about EF. So, my approach to this problem spent a lot of time using <a target="_blank" href="http://www.red-gate.com/products/reflector/index.htm">.NET Reflector</a> to try to understand what the EF designer was doing behind the scenes (this also proved to be a a good way to understand EF better!). The first thing that I wanted to figure out is how EF determines which primitive type to use for each SQL Server type. </p> <p>I downloaded and imported the <a target="_blank" href="http://uscdsql.codeplex.com/releases/view/26397#DownloadId=66068">States</a> data from the <a target="_blank" href="http://uscdsql.codeplex.com/">US Census Data for SQL Server 2008</a> project on Codeplex. Then, I used the Entity Data Model Designer in VS2010 to generate a model based on my database which resulted in an entity without the geometry property. Looking at the XML for the .edmx file, I saw the following:</p> <pre class="csharpcode"><span class="rem"><!--Errors Found During Generation:</span> <span class="rem">warning 6005: The data type 'geometry' is not supported; <br />the column 'geom' in table 'Spatial.dbo.State' was excluded.</span> <span class="rem">--></span><br /> <span class="kwrd"><</span><span class="html">EntityType</span> <span class="attr">Name</span><span class="kwrd">="State"</span><span class="kwrd">></span><br /> <span class="kwrd"> <</span><span class="html">Key</span><span class="kwrd">><br /></span> <span class="kwrd"> <</span><span class="html">PropertyRef</span> <span class="attr">Name</span><span class="kwrd">="StateID"</span> <span class="kwrd">/></span><br /> <span class="kwrd"> </</span><span class="html">Key</span><span class="kwrd">><br /></span> <span class="kwrd"><</span><span class="html">Property</span> <span class="attr">Name</span><span class="kwrd">="StateID"</span> <span class="attr">Type</span><span class="kwrd">="int"</span> <span class="attr">Nullable</span><span class="kwrd">="false"</span> <span class="kwrd">/></span><br /> <span class="kwrd"><</span><span class="html">Property</span> <span class="attr">Name</span><span class="kwrd">="StateName"</span> <span class="attr">Type</span><span class="kwrd">="nvarchar"</span> <span class="attr">Nullable</span><span class="kwrd">="false"</span> <span class="attr">MaxLength</span><span class="kwrd">="50"</span> <span class="kwrd">/><br /></span> <span class="kwrd"></</span><span class="html">EntityType</span><span class="kwrd">></span> </pre> <p> </p> <p>I don’t believe that EF is hating on “geometry” specifically (the 6005 warning). Rather, I think that if the SQL Server type cannot be mapped to a .NET type from the BCL, then it simply does not know how to handle it. Certainly, they don’t want to try to map to a type that is not included in the .NET Framework itself (as would be the case for the Spatial data types).</p> <p>But, what is EF using to determine the mappings?</p> <p>I looked long and hard, but couldn’t quite figure out the mechanism that gets invoked when the model is generated. But, I think the key might lie in the <font face="monospace">Microsoft.VisualStudio.Data.Providers.SqlServer.SqlMappedObjectConverter.GetFrameworkTypeFromNativeType()</font> method:</p> <pre class="csharpcode"><span class="rem">// Disassembly by Reflector</span> <span class="kwrd">protected</span> <span class="kwrd">override</span> Type GetFrameworkTypeFromNativeType(<span class="kwrd">string</span> nativeType)<br />{<br /> <span class="kwrd">switch</span> (<span class="kwrd">this</span>.GetProviderTypeFromNativeType(nativeType))<br /> {<br /> <span class="kwrd">case</span> 0:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">long</span>);<br /><br /> <span class="kwrd">case</span> 1:<br /> <span class="kwrd">case</span> 7:<br /> <span class="kwrd">case</span> 0x13:<br /> <span class="kwrd">case</span> 0x15:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">byte</span>[]);<br /><br /> <span class="kwrd">case</span> 2:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">bool</span>);<br /><br /> <span class="kwrd">case</span> 3:<br /> <span class="kwrd">case</span> 10:<br /> <span class="kwrd">case</span> 11:<br /> <span class="kwrd">case</span> 12:<br /> <span class="kwrd">case</span> 0x12:<br /> <span class="kwrd">case</span> 0x16:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">string</span>);<br /><br /> <span class="kwrd">case</span> 4:<br /> <span class="kwrd">case</span> 15:<br /> <span class="kwrd">case</span> 0x1f:<br /> <span class="kwrd">case</span> 0x21:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(DateTime);<br /><br /> <span class="kwrd">case</span> 5:<br /> <span class="kwrd">case</span> 9:<br /> <span class="kwrd">case</span> 0x11:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">decimal</span>);<br /><br /> <span class="kwrd">case</span> 6:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">double</span>);<br /><br /> <span class="kwrd">case</span> 8:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">int</span>);<br /><br /> <span class="kwrd">case</span> 13:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">float</span>);<br /><br /> <span class="kwrd">case</span> 14:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(Guid);<br /><br /> <span class="kwrd">case</span> 0x10:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">short</span>);<br /><br /> <span class="kwrd">case</span> 20:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">byte</span>);<br /><br /> <span class="kwrd">case</span> 0x20:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(TimeSpan);<br /><br /> <span class="kwrd">case</span> 0x22:<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(DateTimeOffset);<br /> }<br /> <span class="kwrd">return</span> <span class="kwrd">typeof</span>(<span class="kwrd">object</span>);<br />}</pre> <p> </p> <p>For SQL Server, the Native Types come from the System.Data.SqlDbType enumeration:</p> <pre class="csharpcode"><span class="rem">// Disassembly by Reflector</span> <span class="kwrd">public</span> <span class="kwrd">enum</span> SqlDbType<br />{<br /> BigInt = 0,<br /> Binary = 1,<br /> Bit = 2,<br /> Char = 3,<br /> Date = 0x1f,<br /> DateTime = 4,<br /> DateTime2 = 0x21,<br /> DateTimeOffset = 0x22,<br /> Decimal = 5,<br /> Float = 6,<br /> Image = 7,<br /> Int = 8,<br /> Money = 9,<br /> NChar = 10,<br /> NText = 11,<br /> NVarChar = 12,<br /> Real = 13,<br /> SmallDateTime = 15,<br /> SmallInt = 0x10,<br /> SmallMoney = 0x11,<br /> Structured = 30,<br /> Text = 0x12,<br /> Time = 0x20,<br /> Timestamp = 0x13,<br /> TinyInt = 20,<br /> Udt = 0x1d,<br /> UniqueIdentifier = 14,<br /> VarBinary = 0x15,<br /> VarChar = 0x16,<br /> Variant = 0x17,<br /> Xml = 0x19<br />}</pre> <pre class="csharpcode"> </pre> <p>My conclusion here was that if the SQL Server type could only be mapped to System.Object in the BCL (using the GetFrameworkTypeFromNativeType() method), then EF will not support using that field as a property of the entity. This coincides with the fact that to ADO.NET, the Geometry (and Geography) type is a User Defined Type (0x1d).</p> <p><span style="font-style: italic; font-weight: bold;">UPDATE: After all of this, I discovered that in System.Data.Entity.dll, there is a method that is probably a better candidate for what is actually used: System.Data.SqlClient.SqlProviderManifest.GetEdmType(). This method contains a similar switch{} as the code listed above, only it is EDM-specific instead of returning BCL types. Feel free to examine it using Reflector if you're curious about its contents.</span><br /> </p> <h3>The Workaround</h3> <p>Having figured out that piece of the puzzle, I was left with trying to figure out a workaround. If ADO.NET was unable to map a Geometry to a type in the BCL, then could we cast the Geometry as something that would be mappable?</p> <p>SQL Server serializes spatial objects to binary when it saves the data in a table (documented here: <a href="http://msdn.microsoft.com/en-us/library/ee320529.aspx" title="http://msdn.microsoft.com/en-us/library/ee320529.aspx">http://msdn.microsoft.com/en-us/library/ee320529.aspx</a>):</p> <p><a href="http://jasonfollas.com/blog/images/jasonfollas_com/blog/WindowsLiveWriter/SpatialDataandtheEntityFramework_9132/EF-Spatial-1_2.png"><img width="644" height="442" border="0" src="http://jasonfollas.com/blog/images/jasonfollas_com/blog/WindowsLiveWriter/SpatialDataandtheEntityFramework_9132/EF-Spatial-1_thumb.png" alt="EF-Spatial-1" title="EF-Spatial-1" style="border-width: 0px; display: inline;" /></a> </p> <p>This binary data can be used to deserialize (“rehydrate”) the object in .NET code, which is exactly what SQL Server does when it needs to use the spatial objects. So, we just need to find a way for EF to pull these down as a byte array.</p> <p>Looking back at the GetFrameworkTypeFromNativeType function from above, it appears that EF will likely recognize Binary, Image, Timestamp, and Varbinary all as SQL Server types that need to map to byte arrays. Perfect!</p> <p>So, by creating a view in SQL Server that casts the Geometry column as a Varbinary(MAX), EF would recognize it as a type that could be mapped as an entity’s property.</p> <pre class="csharpcode"><span class="kwrd">CREATE</span> <span class="kwrd">VIEW</span> vStates<br /><span class="kwrd">AS</span> <span class="kwrd">SELECT</span> StateID<br /> , StateName<br /> , <span class="kwrd">CAST</span>(geom <span class="kwrd">AS</span> VARBINARY(<span class="kwrd">MAX</span>)) <span class="kwrd">AS</span> geom<br /> <span class="kwrd">FROM</span> dbo.State</pre> <p> </p> <p><em>Note: Julie had come up with this same solution at the same time, as our emails crossed paths reporting to one another.</em></p> <p>Regenerating the EF model (using this view instead of the table) proved my assumption: the “geom” column now appeared as a Binary property of the vStates entity.</p> <p>However, we’re not quite done yet. The point of this exercise was to get an instance of the spatial type to use in our .NET application. To do that, the Read(BinaryReader) instance method on SqlGeometry (or SqlGeography) must be invoked (using a MemoryStream as the intermediate between the byte[] and the BinaryReader).</p> <p>The entire logic to retrieve the contents of the table and instantiate one of the Spatial types is as follows:</p> <pre class="csharpcode">var entities = <span class="kwrd">new</span> SpatialEntities();<br />var vStates = entities.vStates;<br /><br /><span class="rem">// pull one of the entities from the collection</span> var geo2 = vStates.ToArray()[16]; <br />var sqlGeom = <span class="kwrd">new</span> Microsoft.SqlServer.Types.SqlGeometry();<br /><br /><span class="rem">// Deserialize the bytes to rehydrate this Geometry instance</span> <span class="kwrd">using</span> (var stream = <span class="kwrd">new</span> System.IO.MemoryStream(geo2.geom))<br />{<br /> <span class="kwrd">using</span> (var rdr = <span class="kwrd">new</span> System.IO.BinaryReader(stream))<br /> {<br /> sqlGeom.Read(rdr);<br /> }<br />}<br /><br /><span class="rem">// Now let's prove that we have it. Dump WKT to Debug.</span> <br />System.Diagnostics.Debug.Write(sqlGeom.ToString());</pre> <style type="text/css"><![CDATA[ .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }]]></style> <p> </p> <p>Output:</p> <p>GEOMETRYCOLLECTION (LINESTRING (-99.530670166015625 39.132522583007812, -99.530670166015625 39.13250732421875), LINESTRING (-99.791290283203125 39.131988525390625, -99.791290283203125 39.131973266601562), …</p> <p>So it worked!</p> <p>Finally, an extension method would make this code a bit more general purpose:</p> <pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">static</span> <span class="kwrd">class</span> Extension<br />{<br /> <span class="kwrd">public</span> <span class="kwrd">static</span> Microsoft.SqlServer.Types.SqlGeometry AsSqlGeometry(<span class="kwrd">this</span> <span class="kwrd">byte</span>[] binary)<br /> {<br /> var ret = <span class="kwrd">new</span> Microsoft.SqlServer.Types.SqlGeometry();<br /><br /> <span class="kwrd">using</span> (var stream = <span class="kwrd">new</span> System.IO.MemoryStream(binary))<br /> {<br /> <span class="kwrd">using</span> (var rdr = <span class="kwrd">new</span> System.IO.BinaryReader(stream))<br /> {<br /> ret.Read(rdr);<br /> }<br /> }<br /><br /> <span class="kwrd">return</span> ret;<br /> }</pre> <pre class="csharpcode">} </pre> <p /><style type="text/css"><![CDATA[ .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, "Courier New", courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }]]></style> <p>The test code above then becomes a bit more readable after the refactoring:</p> <pre class="csharpcode">var entities = <span class="kwrd">new</span> SpatialEntities();<br />var vStates = entities.vStates;<br /><br /><span class="rem">// pull one of the entities from the collection</span> var geo2 = vStates.ToArray()[16];<br />var sqlGeom = geo2.geom.AsSqlGeometry();<br /><span class="rem"><br />// Now let's prove that we have it. Dump WKT to Debug.</span> <br />System.Diagnostics.Debug.Write(sqlGeom.ToString());</pre> <p> </p> <p>Helpful information:</p> <ul> <li><a target="_blank" href="http://thedatafarm.com/blog/data-access/yes-you-can-read-and-probably-write-spatial-data-with-entity-framework/">Julie’s blog post on this exercise</a></li> <li><a target="_blank" href="http://jasonfollas.com/blog/archive/2009/02/21/knowledge-4.aspx">My blog post on obtaining the SQL Server Types assembly</a></li> <li><a href="http://social.msdn.microsoft.com/Forums/en-US/sqlspatial/thread/a77783e9-572e-4841-b4da-f2461296d596">SQL Spatial Forum post describing this same solution (I discovered this after I had put in the effort)</a></li> </ul><img src="http://jasonfollas.com/blog/aggbug/74.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2010/02/14/spatial-data-and-the-entity-framework.aspx Sun, 14 Feb 2010 20:14:47 GMT http://jasonfollas.com/blog/archive/2010/02/14/spatial-data-and-the-entity-framework.aspx#feedback 2 http://jasonfollas.com/blog/comments/commentRss/74.aspx http://jasonfollas.com/blog/services/trackbacks/74.aspx Speaking at Microsoft SQL Server User Group in Columbus, June 5 http://jasonfollas.com/blog/archive/2009/05/20/speaking-at-microsoft-sql-server-user-group-in-columbus-june.aspx <p><font face="Arial">On June 5th, I'll be in Columbus (Ohio) speaking to the <a target="_blank" href="https://www.resultdata.com/training/schedule/register.aspx?id=198020">Microsoft SQL Server User Group</a> (Result Data) about using the XML data type in SQL Server.</font></p> <p><font face="Arial"><strong>XML Capabilities of SQL Server 2008</strong></font></p> <p><font face="Arial">Jason Follas, Microsoft MVP for SQL Server, will lead a discussion on the XML capabilities of SQL Server 2008. XML-based data is very prevalent in today's data world to the point that every database developer should know how to work with XML. Middle-tier translation of XML into relational data may be inflexible, and often leads to a loss of fidelity over time. One solution is to move the XML handling into the database itself in order to improve overall integration. SQL Server 2008 provides powerful XML processing features that allow easy access to data stored within XML without sacrificing any of the original fidelity.</font></p> <p><font face="Arial"><strong>Partitioning in SQL Server 2008</strong></font></p> <p><font face="Arial">Andy Thissen, Database Administrator at BMW Financial Services, will lead a discussion of the use of partitioning in SQL Server 2008. <br /> <br /> Agenda<br /> <br /> 12:45 Check-In<br /> 1:15 Introduction & Announcements<br /> 1:30 XML Capabilities of SQL Server 2008<br /> 2:30 BREAK<br /> 2:45 Partitioning in SQL Server 2008<br /> 3:30 Q&A<br /> 3:45 Raffle – Door Prize </font></p> <p><font face="Arial"><a href="https://www.resultdata.com/training/schedule/register.aspx?id=198020">https://www.resultdata.com/training/schedule/register.aspx?id=198020</a></font></p><img src="http://jasonfollas.com/blog/aggbug/69.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2009/05/20/speaking-at-microsoft-sql-server-user-group-in-columbus-june.aspx Wed, 20 May 2009 15:00:05 GMT http://jasonfollas.com/blog/archive/2009/05/20/speaking-at-microsoft-sql-server-user-group-in-columbus-june.aspx#feedback http://jasonfollas.com/blog/comments/commentRss/69.aspx http://jasonfollas.com/blog/services/trackbacks/69.aspx For Jeff: Spatial Querying http://jasonfollas.com/blog/archive/2009/03/04/for-jeff-spatial-querying.aspx <p>A quick sample of usage as requested by my friend <a href="http://www.mcwherter.net/blog/">Jeff McWherter</a>:</p> <p>Querying for all data in a table where the "zip code polygon" is within 20 miles of a geocode point (must convert 20 miles to meters in the predicate)::</p> <pre class="sql" name="code">select * from dbo.fe_2007_us_zcta500 where Boundary.STDistance('POINT(-79.8884595930576 43.2609696686268)') < (20 * 1609.344) </pre> <img src="http://jasonfollas.com/blog/aggbug/67.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2009/03/04/for-jeff-spatial-querying.aspx Wed, 04 Mar 2009 23:08:55 GMT http://jasonfollas.com/blog/archive/2009/03/04/for-jeff-spatial-querying.aspx#feedback 1 http://jasonfollas.com/blog/comments/commentRss/67.aspx http://jasonfollas.com/blog/services/trackbacks/67.aspx How To Force SSRS To Use Latest Data http://jasonfollas.com/blog/archive/2009/02/25/how-to-force-ssrs-to-use-latest-data.aspx <p>SQL Server Reporting Services is great in the fact that it comes with a handful of different rendering (export) formats out of the box. If a particular report is intended to be paginated and printed, then from my web applications, I will often link to the report with a parameter instructing the render format to be PDF - this way, the user is never taken to the report viewer (they just get an Open/Save As dialog for the PDF itself).</p> <p>The URL for such a link may resemble the following:</p> <p><font face="Arial">http://server/ReportServer?/path/reportname&rs:Command=Render&rc:LinkTarget=_blank&<strong><u>rs:Format=PDF</u></strong>&SomeParameterValue=1</font></p> <p>There are other interesting things that you can do with the rs:Format parameter. For instance, say that you create a tiny report that only contains a chart. Why not have this report rendered as an image, and then you can show this chart on your website (i.e., no extra charting component would be necessary on your web server). Simply specify the URL to your report in an IMG tag's SRC property, and include the following parameters in that URL:</p> <p>rs:Format=IMAGE&rc:OutputFormat=PNG (or GIF, or JPG, etc)</p> <p>One problem with not going through the default web-based report viewer, though: the report server likes to cache the output for your browser session. Therefore, if the underlying data changes, re-running these reports will not show the updated information unless you close your browser and hit the page again.</p> <p>After a bit of headscratching and searching on the Internet, I found an easy solution that can be used to always force the reports to query the database. Simply include the following parameter in your report's URL:</p> <p>rs:ClearSession=true</p> <p>Of course, you will take more of a hit on your database (because the report will always requery its datasets). But, perhaps in your solution, you can selectively use this parameter if the user wants to force a refresh of a report that is automatically rendered to PDF or an image - that's a design decision that you'll have to make. </p> <p>I'm just happy that a mechanism exists to get around the default behavior!</p> <p> </p> <p><font face="Arial" /></p><img src="http://jasonfollas.com/blog/aggbug/66.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2009/02/25/how-to-force-ssrs-to-use-latest-data.aspx Wed, 25 Feb 2009 19:02:27 GMT http://jasonfollas.com/blog/archive/2009/02/25/how-to-force-ssrs-to-use-latest-data.aspx#feedback 1 http://jasonfollas.com/blog/comments/commentRss/66.aspx http://jasonfollas.com/blog/services/trackbacks/66.aspx Knowledge++ [4] http://jasonfollas.com/blog/archive/2009/02/21/knowledge-4.aspx <p>I recently developed a spatially-aware .NET application that did not use SQL Server 2008 as the backend (this enterprise was still on SS2005, but we needed the spatial support in the application today). While the application worked properly on my laptop, it was a huge failboat when deployed to the server environment.</p> <p>I had previously posted that you can get the Microsoft.SqlServer.Types library from MS Downloads, but it turns out that this alone is not sufficient to allow your application to run. You also need to ensure that the SQL Server 2008 Native Client is also installed (regardless of whether you're accessing a SS2008 instance or not). <em><strong><font color="#ff0000">Update!</font> </strong>You actually don't... read below.</em></p> <p>Both the Types library and the Native Client can be downloaded from the following:</p> <p><font face="Arial"><a href="http://www.microsoft.com/downloads/details.aspx?FamilyID=228de03f-3b5a-428a-923f-58a033d316e1&DisplayLang=en">http://www.microsoft.com/downloads/details.aspx?FamilyID=228de03f-3b5a-428a-923f-58a033d316e1&DisplayLang=en</a></font></p> <p><em>My discovery source: <font face="Arial"><a href="https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=355402&wa=wsignin1.0">https://connect.microsoft.com/SQLServer/feedback/ViewFeedback.aspx?FeedbackID=355402&wa=wsignin1.0</a></font></em></p> <p><strong>UPDATE: </strong>Per <a href="http://blogs.msdn.com/isaac/default.aspx">Isaac Kunen</a> (in this blog post's comments as well as offline discussion), the missing component from the Types library is simply an updated version of the C Runtime. The fix of using the Native Client is a hack in this case because its MSI actually installs the updated CRT (which the MSI for the Types library should have done also - it's a goof that MS Downloads hasn't been updated with an updated version of the Types API after the above Connect feedback was answered...</p> <p>The Microsoft Visual C++ 2008 redistributable by itself can be downloaded from:</p> <p><font face="Arial"><a href="http://www.microsoft.com/downloads/details.aspx?FamilyID=A5C84275-3B97-4AB7-A40D-3802B2AF5FC2&displaylang=en">http://www.microsoft.com/downloads/details.aspx?FamilyID=A5C84275-3B97-4AB7-A40D-3802B2AF5FC2&displaylang=en</a></font></p> <p> </p><img src="http://jasonfollas.com/blog/aggbug/63.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2009/02/21/knowledge-4.aspx Sun, 22 Feb 2009 02:45:26 GMT http://jasonfollas.com/blog/archive/2009/02/21/knowledge-4.aspx#feedback 3 http://jasonfollas.com/blog/comments/commentRss/63.aspx http://jasonfollas.com/blog/services/trackbacks/63.aspx Knowledge++ [3] http://jasonfollas.com/blog/archive/2009/01/29/knowledge-3.aspx <p>When you write a query that SELECTs a SQL Server 2008 Spatial type, the returned result is a binary value. Without closely examining the bytes, I just assumed that this was WKB (Well-Known Binary).</p> <p>Well, as it turns out, the bytes that are returned are simply a serialized version of the .NET object (the UDT has a Write() method that it calls internally to serialize the object in binary). This is NOT directly compatible with WKB.</p> <p><em>My discovery source: <font face="Arial"><a href="http://social.msdn.microsoft.com/Forums/en-US/sqlspatial/thread/40ee9466-d7bf-4340-b295-53217ae5128e">http://social.msdn.microsoft.com/Forums/en-US/sqlspatial/thread/40ee9466-d7bf-4340-b295-53217ae5128e</a></font></em></p> <img src="http://jasonfollas.com/blog/aggbug/62.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2009/01/29/knowledge-3.aspx Thu, 29 Jan 2009 14:32:28 GMT http://jasonfollas.com/blog/archive/2009/01/29/knowledge-3.aspx#feedback http://jasonfollas.com/blog/comments/commentRss/62.aspx http://jasonfollas.com/blog/services/trackbacks/62.aspx Using SQL Server Spatial Objects as ADO.NET Parameter Values http://jasonfollas.com/blog/archive/2008/12/11/using-ado.net-with-sql-server-spatial-objects.aspx <p>I've <a target="_blank" href="http://jasonfollas.com/blog/archive/2008/06/23/sql-server-2008-spatial-data-part-8.aspx">previously mentioned</a> that the SQL Server 2008 Spatial data types are freely available for use in your .NET applications, regardless of whether you have SQL Server 2008 or not. This allows you to incorporate some powerful spatial capabilities right into your application. </p> <p><em>(Look for "<font face="Arial">Microsoft SQL Server System CLR Types" on this page: <font face="Arial"><a href="http://www.microsoft.com/downloads/details.aspx?FamilyID=228DE03F-3B5A-428A-923F-58A033D316E1&displaylang=en">http://www.microsoft.com/downloads/details.aspx?FamilyID=228DE03F-3B5A-428A-923F-58A033D316E1&displaylang=en</a> )</font></font></em></p> <p>However, in most usage scenarios, there will come a time when you have an instance of a SQL Server spatial object in your .NET application, and need to commit it to your SQL Server 2008 database. How would you do this, without losing fidelity or resorting to serialization of the object to WKT first?</p> <p>The solutions is to create a Parameter object of type System.Data.SqlDbType.Udt. Then set the UdtTypeName parameter to the SQL Server-recognized type name (i.e., for SqlGeometry, you would simply use Geometry).</p> <p>The following code demonstrates executing an UPDATE statement that sets the value of a Spatial field to a newly constructed object.</p> <pre class="c#" name="code">using (SqlConnection conn = new SqlConnection("Server=.;Integrated Security=true;Initial Catalog=scratch")) { using (SqlCommand cmd = new SqlCommand("UPDATE fe_2007_us_zcta500 SET Boundary=@boundary WHERE id=@id", conn)) { SqlParameter id = cmd.Parameters.Add("@id", System.Data.SqlDbType.Int); SqlParameter boundary = cmd.Parameters.Add("@boundary", System.Data.SqlDbType.Udt); boundary.UdtTypeName = "geometry"; SqlGeometry geom = SqlGeometry.Parse("POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))"); boundary.Value = geom; id.Value = 123; conn.Open(); cmd.ExecuteNonQuery(); conn.Close(); } } </pre><img src="http://jasonfollas.com/blog/aggbug/58.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2008/12/11/using-ado.net-with-sql-server-spatial-objects.aspx Thu, 11 Dec 2008 19:04:12 GMT http://jasonfollas.com/blog/archive/2008/12/11/using-ado.net-with-sql-server-spatial-objects.aspx#feedback http://jasonfollas.com/blog/comments/commentRss/58.aspx http://jasonfollas.com/blog/services/trackbacks/58.aspx Flashy SQL Server 2008 Site http://jasonfollas.com/blog/archive/2008/12/08/flashy-sql-server-2008-site.aspx <p><a target="_blank" href="http://www.microsoft.com/servers/sql/2008/sqlserverenergy/en/us/default.aspx"><img alt="SQL Server 2008: It's a new form of energy." hspace="5" align="right" border="0" src="http://jasonfollas.com/blog/images/jasonfollas_com/blog/9/o_SQL2008Energy.png" /></a>As a matter of principal, not to mention terms of service, I don't often click on ads that appear on <a href="http://jasonfollas.com/blog">my own blog</a>. But, when I see an ad on my site that interests me, I follow it (I believe Google knows who I am, so I don't get "credit" for that click).</p> <p>Such was the case the other day when I saw a flashy plasma-lamp-like animated advertisement in <a href="http://jasonfollas.com/blog">my site</a>'s sidebar with the words "They're not just new forms of data... It's a new form of energy. SQL Server 2008".</p> <p>I won't ask you to click on the ad from <a href="http://jasonfollas.com/blog">my site</a>, but give the destination site a whirl to learn about the cool things that SQL Server 2008 can do (including something that I'm very interested in: spatial querying):</p> <p><font face="Arial"><a target="_blank" href="http://www.microsoft.com/servers/sql/2008/sqlserverenergy/en/us/default.aspx">http://www.microsoft.com/servers/sql/2008/sqlserverenergy/en/us/default.aspx</a></font></p> <p> </p> <p> </p><img src="http://jasonfollas.com/blog/aggbug/57.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2008/12/08/flashy-sql-server-2008-site.aspx Mon, 08 Dec 2008 20:14:40 GMT http://jasonfollas.com/blog/archive/2008/12/08/flashy-sql-server-2008-site.aspx#feedback http://jasonfollas.com/blog/comments/commentRss/57.aspx http://jasonfollas.com/blog/services/trackbacks/57.aspx SqlGeography: Ring Orientation of Polygon Interior Rings (Holes) http://jasonfollas.com/blog/archive/2008/11/25/sqlgeography-ring-orientation-of-polygon-interior-rings-holes.aspx <p>I have mentioned before how the Ring Orientation for the exterior ring of a Polygon is significant when instantiating a SqlGeography object. In this case, a Counter-Clockwise orientation is required so that as an observer walks along the path, the interior of the Polygon is always to their left.</p> <p><img alt="Ring Orientation for SqlGeography" hspace="5" align="right" src="http://jasonfollas.com/blog/images/jasonfollas_com/blog/9/r_RingOrientation.PNG" />But, what I have never really seen documented (or paid attention to, at least) is the fact that the interior rings, or holes, of a Polygon also have specific Ring Orientation requirements. </p> <p>In keeping with the "Left-handed" rule, interior rings must be defined in a Clockwise manner - the opposite orientation of the shape's exterior ring. This is because holes within a Polygon are considered to be part of the exterior of the shape, so the observer walking in a Clockwise direction is still keeping the Polygon's interior to their left.</p> <p>(I should note here that the Ring Orientation for SqlGeography is the exact opposite of ESRI's ShapeFile format, which is why Ring Orientation has been on my mind for the past few days).</p> <p> </p><img src="http://jasonfollas.com/blog/aggbug/55.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2008/11/25/sqlgeography-ring-orientation-of-polygon-interior-rings-holes.aspx Tue, 25 Nov 2008 14:14:18 GMT http://jasonfollas.com/blog/archive/2008/11/25/sqlgeography-ring-orientation-of-polygon-interior-rings-holes.aspx#feedback 3 http://jasonfollas.com/blog/comments/commentRss/55.aspx http://jasonfollas.com/blog/services/trackbacks/55.aspx Spatial: Determining Ring Orientation http://jasonfollas.com/blog/archive/2008/11/24/spatial-determining-ring-orientation.aspx <p>A Ring is a list of points such that the starting point and ending point are the same (forming a closed shape). The order the you define the points that make up a Ring - known as Ring Orientation - is significant, for various data formats (including SQL Server's Geography type) imply special meaning for rings that are defined in a clockwise manner as opposed to a counter-clockwise manner. </p> <p>Given a list of points with no additional context, it can be difficult to determine the Ring Orientation being used. </p> <p>For example, suppose that you have a generic list of points that represent the boundary of a postal code, and that you wish to use these points in order to construct a Polygon instance using the SqlGeography type. SqlGeography happens to use "Left-handed" ordering, so that as an observer walks along the set of points in the order defined, the "inside" of the polygon is always to their left. This also implies that the exterior ring of a Polygon is defined in a counter-clockwise manner.</p> <p>If you try to define a polygon with an area greater than a single hemisphere (this is a nice way to say "if you screw up and use the wrong orientation"), then the SqlGeography type will throw an exception. So, aside from using Try-Catch, what can you do?</p> <p>While researching solutions to this problem, I stumbled upon a paper entitled "<a href="http://www.engr.colostate.edu/~dga/dga/papers/point_in_polygon.pdf">A Winding Number and Point-in-Polygon Algorithm</a>" from the Colorado State University. It turns out that a simple algorithm with O(n) complexity can be used to determine if a point is within a Polygon, and a side effect also provides the Ring Orientation. The key to this algorithm is determining the trend of the ring at each crossing of an axis.</p> <p>Since I was only interested in Ring Orientation (and not point enclosure detection), I didn't need to use this particular algorithm. Instead, I took inspiration from the winding concept, and created a simpler derivative algorithm:</p> <p><img alt="Visual example of how to determine ring orientation at the extreme left point" hspace="5" align="right" src="http://jasonfollas.com/blog/images/jasonfollas_com/blog/9/o_orientation_example.png" /> </p> <ol> <li>Iterate the point collection and determine the extreme "left" and "right" points </li> <li>Normalize the line segments connected to these points so that they each have the same "X" dimension length </li> <li>Compare the "Y" values of the normalized segments to establish the trend through that extreme point (i.e., is the "previous" segment above or below the "next" segment) </li> <li>In the spirit of the Winding algorithm, use opposite orientations for the left and right points so that the results coincide with one another </li> <li>A negative result (negative indicates Clockwise orientation, positive result indicates Counter-Clockwise orientation, and a result of zero would be undefined </li> </ol> <p>I've actually written (and posted) several versions of this algorithm, each time discovering some edge case exception that would cause me to take down the post and rewrite the algorithm. I believe the code below works for all simple polygons on a Cartesian coordinate system (read: I have more testing to see if this will work with an ellipsoidal model, like SqlGeography). </p> <p><em>Note: The following code is generic in nature, and as such, I've defined my own Point structure instead of using a SqlGeometry or SqlGeography, etc. </em></p> <pre class="c#" name="code">struct Point { public double X { get; set; } public double Y { get; set; } } enum RingOrientation : int { Unknown = 0, Clockwise = -1, CounterClockwise = 1 }; RingOrientation Orientation(Point[] points) { // Inspired by http://www.engr.colostate.edu/~dga/dga/papers/point_in_polygon.pdf // This algorithm is to simply determine the Ring Orientation, so to do so, find the // extreme left and right points, and then check orientation if (points.Length < 4) { throw new ArgumentException("A polygon requires at least 4 points."); } if (points[0].X != points[points.Length - 1].X || points[0].Y != points[points.Length - 1].Y) { throw new ArgumentException("The array of points is not a polygon. The first and last point must be identical."); } int rightmostIndex = 0; int leftmostIndex = 0; for (int i = 1; i < points.Length; i++) { if (points[i].X < points[leftmostIndex].X) { leftmostIndex = i; } if (points[i].X > points[rightmostIndex].X) { rightmostIndex = i; } } Point p0; // Point before the extreme Point p1; // The extreme point Point p2; // Point after the extreme double m; // Holds line slope double lenP2x; // Length of the P1-P2 line segment's delta X double newP0y; // The Y value of the P1-P0 line segment adjusted for X=lenP2x RingOrientation left_orientation; RingOrientation right_orientation; // Determine the orientation at the Left Point if (leftmostIndex == 0) p0 = points[points.Length - 2]; else p0 = points[leftmostIndex - 1]; p1 = points[leftmostIndex]; if (leftmostIndex == points.Length - 1) p2 = points[1]; else p2 = points[leftmostIndex + 1]; m = (p1.Y - p0.Y) / (p1.X - p0.X); if (double.IsInfinity(m)) { // This is a vertical line segment, so just calculate the dY to // determine orientation left_orientation = (RingOrientation)Math.Sign(p0.Y - p1.Y); } else { lenP2x = p2.X - p1.X; newP0y = p1.Y + (m * lenP2x); left_orientation = (RingOrientation)Math.Sign(newP0y - p2.Y); } // Determine the orientation at the Right Point if (rightmostIndex == 0) p0 = points[points.Length - 2]; else p0 = points[rightmostIndex - 1]; p1 = points[rightmostIndex]; if (rightmostIndex == points.Length - 1) p2 = points[1]; else p2 = points[rightmostIndex + 1]; m = (p1.Y - p0.Y) / (p1.X - p0.X); if (double.IsInfinity(m)) { // This is a vertical line segment, so just calculate the dY to // determine orientation right_orientation = (RingOrientation)Math.Sign(p1.Y - p0.Y); } else { lenP2x = p2.X - p1.X; newP0y = p1.Y + (m * lenP2x); right_orientation = (RingOrientation)Math.Sign(p2.Y - newP0y); } if (left_orientation == RingOrientation.Unknown) { return right_orientation; } else { return left_orientation; } } void Test() { // Simple triangle - left extreme point is vertically "in between" line segments Point[] points = new Point[] { new Point(5,-1), new Point(0,0), new Point(5,1), new Point(5,-1) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); // Case where both line segments are above the left extreme point points = new Point[] { new Point(2,1), new Point(0,0), new Point(1,1), new Point(2,1) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); // Case where both line segments are below the left extreme point points = new Point[] { new Point(2,-1), new Point(0,0), new Point(1,-1), new Point(2,-1) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); // Case where line segment is vertical (slope cannot be determined) points = new Point[] { new Point(0,0), new Point(0,1), new Point(1,1), new Point(1,0), new Point(0,0) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); // Case where angle thru left extreme point is a right angle points = new Point[] { new Point(0,0), new Point(1,1), new Point(1,-1), new Point(0,0) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); // Real-world case from a SHP file points = new Point[] { new Point(-156.92467299999998,20.738695999999997), new Point(-156.924636,20.738822), new Point(-156.924608,20.73894), new Point(-156.92458,20.739082), new Point(-156.92460599999998,20.739234), new Point(-156.924551,20.739326), new Point(-156.924507,20.739241999999997), new Point(-156.924482,20.739082), new Point(-156.924466,20.738854999999997), new Point(-156.924387,20.738602999999998), new Point(-156.924308,20.738325), new Point(-156.924239,20.738063999999998), new Point(-156.92424,20.737887999999998), new Point(-156.924285,20.737811999999998), new Point(-156.924475,20.73762), new Point(-156.92458299999998,20.737603999999997), new Point(-156.924754,20.737579), new Point(-156.924851,20.737731), new Point(-156.924956,20.738101), new Point(-156.924909,20.738343999999998), new Point(-156.924818,20.738487), new Point(-156.92467299999998,20.738695999999997) }; System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.Clockwise); Array.Reverse(points); System.Diagnostics.Debug.Assert(Orientation(points) == RingOrientation.CounterClockwise); }</pre><img src="http://jasonfollas.com/blog/aggbug/54.aspx" width="1" height="1" /> Jason Follas http://jasonfollas.com/blog/archive/2008/11/24/spatial-determining-ring-orientation.aspx Mon, 24 Nov 2008 18:00:46 GMT http://jasonfollas.com/blog/archive/2008/11/24/spatial-determining-ring-orientation.aspx#feedback http://jasonfollas.com/blog/comments/commentRss/54.aspx http://jasonfollas.com/blog/services/trackbacks/54.aspx