AdaTheDev Blog

Bulk Load To Azure Table Storage Using Azure Function Scale Out

adathedev@gmail.com (Adrian Hills) — Mon, 14 Feb 2022 00:00:00 +0000

In this post, we’ll work though how we can maximize bulk insert throughput to Azure Table Storage using Azure Functions in a consumption plan.

Monitoring Azure Function Consumption Plan Instance Scale Out

adathedev@gmail.com (Adrian Hills) — Sat, 29 Jan 2022 00:00:00 +0000

With the Consumption Plan hosting model for Azure Functions, we can take advantage of event-driven scaling to automatically spin up extra instances of a Function app based on load. In this post, we’ll cover how you can view this in real-time and also how to chart the scaled-out instances retrospectively over time.

E2E Testing Azure Functions

adathedev@gmail.com (Adrian Hills) — Mon, 10 Jan 2022 00:00:00 +0000

Scenario Recently I was working on a solution which included multiple Azure Function apps, with unit tests (xUnit) giving rapid feedback for discrete components across the solution in isolation. But the next step was to create some automated end-to-end (E2E) tests that exercised the behaviours of the solution as a whole. The goal was to be able to get that feedback as early on in the development process as possible which meant being able to run those E2E tests against a local build on a dev machine when needed with minimal hassle, and in an Azure DevOps CI/CD pipeline, without requiring the Azure Functions to be deployed first.

Troubleshooting Azure Function Consumption Plan Scaling

adathedev@gmail.com (Adrian Hills) — Fri, 17 Dec 2021 13:02:00 +0000

I recently encountered an issue with an Azure Function Event Hub trigger running on a Linux Consumption Plan, whereby events stopped being consumed by the trigger once it went to sleep. I learned a lot about consumption plan scaling and diagnosing what was going on, so in this post I’m going to share the key takeaways that I picked up as it was a bit of a journey. Background: Azure Function Event Hub trigger stopped processing events This was in a low-throughput scenario for testing - so the rate of events coming into the Event Hub was low/sporadic.

Samsung Gear Sport vs SWIMTAG - Swimathon 2018

adathedev@gmail.com (Adrian Hills) — Sun, 29 Apr 2018 11:51:00 +0100

This post is a bit different to my normal ones in that it’s not my usual ramblings about something software development related. Instead, it’s a cross between a write-up of my taking part in Swimathon 2018, and a side-by-side comparison of how my swim was tracked by 2 different bits of kit: a Samsung Gear Sport and SWIMTAG. For those that sponsored me, this is proof that I held up my side of the deal!

Getting started with SignalR

adathedev@gmail.com (Adrian Hills) — Tue, 11 Oct 2016 22:56:00 +0100

Follows on from previous post: Real-time SignalR and Angular 2 dashboard I had a couple of questions from someone who forked the project on GitHub and is using it as the basis for their own custom dashboard, which reminded me that there are a few key things I picked up on my learning journey that I should share. These are primarily focused around SignalR aspects. An instance of a SignalR hub is created for each request A single instance is not shared across all clients connected to that hub.

Real-time SignalR and Angular 2 dashboard

adathedev@gmail.com (Adrian Hills) — Fri, 07 Oct 2016 09:14:00 +0100

tl;dr Check out my new SignalRDashboard GitHub repo Followup post: Getting started with SignalR For a while now, I’ve had that burning curiosity to do more than just the bit of hacking around with SignalR that I’d previously done. I wanted to actually start building something with it. If you’re not familiar with it, SignalR is a: …library for ASP.NET developers that makes developing real-time web functionality easy. It provides a simple way to have two-way communication between the client and server; the server can push updates/data to any connected client.

dm_exec_query_plan returning NULL query plan

adathedev@gmail.com (Adrian Hills) — Tue, 12 Aug 2014 16:03:00 +0100

I recently hit a scenario (SQL Server 2012 Standard, 11.0.5058) where I was trying to pull out the execution plan for a stored procedure from the plan cache, but the query shown below was returning a NULL query plan:

70-486 Developing ASP.NET MVC 4 Web Applications

adathedev@gmail.com (Adrian Hills) — Thu, 19 Sep 2013 09:59:00 +0100

Last week I passed the 70-486 Microsoft exam - Developing ASP.NET MVC 4 Web Applications, so thought I’d knock up a quick post on my experience and what materials I found useful as part of my preparation. Going into this exam, I had just over a year and a half’s commercial experience using ASP.NET MVC 2 & 3. Before that, I had experience in ASP.NET and prior to that, when dinosaurs still roamed, classic ASP.

SQL Server 2008 R2 in-place upgrade error

adathedev@gmail.com (Adrian Hills) — Wed, 05 Jun 2013 21:18:00 +0100

Today I encountered the following error during the process of performing an in-place upgrade of a SQL Server 2008 instance to 2008 R2: The specified user 'someuser@somedomain.local' does not exist I wasn’t initially sure what that related to - I hadn’t specified that account during the upgrade process so I went looking in the Services managemement console. The SQL Server service for the instance I was upgrading was configured to “Log On As” that account and it had been happily running prior to the upgrade.

.NET Project File Analyser

adathedev@gmail.com (Adrian Hills) — Thu, 21 Mar 2013 09:53:00 +0000

I’ve started knocking together a little app to automate the process of trawling through a folder structure and checking .NET project files (C# .csproj currently) to extract some info out of them. The DotNetProjectFileAnalyser repo is up on GitHub. I’ve been working against Visual Studio 2010 project files, but could well work for other versions assuming the project file structure is the same for the elements it currently looks at - I just haven’t tried as yet.

GB Post Code Importer Conversion Accuracy Fix

adathedev@gmail.com (Adrian Hills) — Wed, 13 Mar 2013 23:09:00 +0000

In a post last year (Ordnance Survey Data Importer Coordinate Conversion Accuracy) I looked into an accuracy issue with the conversion process within the GeoCoordConversion DLL that I use in this project (blog post). Bottom line, was that it was a minor with an average inaccuracy of around 2.5 metres and a max of ~130 metres by my reckoning. I’ve since had a few requests asking if I can supply an updated GeoCoordConversion DLL with fixes to the calculations.

SQL Server Table Designer Bug With Filtered Unique Index

adathedev@gmail.com (Adrian Hills) — Fri, 08 Mar 2013 13:11:00 +0000

A colleague was getting a duplicate key error when trying to add a new column to a table via the Table Designer in SQL Server Management Studio (2008R2 - not tested on other versions), despite there being no violating data in the table. After a bit of digging around, I tracked the problem down to what appears to be a bug in Table Designer when there is a unique, filtered index in place on the table and the table is being recreated (i.

MongoDB ASP.NET Session Store Provider v1.1.0

adathedev@gmail.com (Adrian Hills) — Sun, 03 Mar 2013 22:05:00 +0000

Since I created the MongoDB ASP.NET Session State Store Provider (v1.0.0), a few things have moved on in the MongoDB C# Driver. I’ve pushed a number of changes up to the project on GitHub (which I’ve incremented to v1.1.0), so it now uses v1.7.0.4714 of the driver. There is no change to way it is configured in web.config, so if you are using v1.0.0 of my provider it should be painless.

Importing twitter archive into ElasticSearch

adathedev@gmail.com (Adrian Hills) — Sun, 10 Feb 2013 19:53:00 +0000

I recently downloaded my full history of tweets from Twitter (via the “Request your archive” button on the Settings page), which gives the data in CSV and JSON formats. The download comes with a simple web UI so you can browse locally, all your historical tweets, as well as perform basic searches. When I tried performing a wildcard search, it resulted in an unresponsive script warning and nobbled the browser tab.

Configuring Azure Storage Emulator SQL Server Instance

adathedev@gmail.com (Adrian Hills) — Mon, 28 May 2012 10:20:00 +0100

If you’re using Windows Azure Storage, you are almost certainly going to be running the storage emulator during development, instead of working directly against your storage account up in the cloud. This emulator (which comes in the Windows Azure SDK - see the “other” category), allows you to test locally against local instances of Blob, Queue and Table services. As per the Overview of Running a Windows Azure Application with Storage Emulator reference, the emulator needs a local SQL Server instance.

Error Creating Azure Blob Storage Container

adathedev@gmail.com (Adrian Hills) — Thu, 24 May 2012 09:19:00 +0100

I received a rather…vague…error when trying out a bit of .NET code to connect to a Windows Azure Blob Storage account, and create a new container in which to store some blobs. The code CloudStorageAccount storageAccount = CloudStorageAccount.Parse( "DefaultEndpointsProtocol=https;AccountName=REMOVED;AccountKey=REMOVED"); CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient(); CloudBlobContainer container = blobClient.GetContainerReference("TestContainer1"); container.CreateIfNotExist(); // This error'd The error A StorageClientException was thrown saying “One of the request inputs is out of range.”. An inner WebException showed that “The remote server returned an error: (400) Bad Request.

ASP.NET MVC Performance Profiling

adathedev@gmail.com (Adrian Hills) — Wed, 23 May 2012 09:33:00 +0100

Building up a profile of how a web application functions, all the database interactions that take place and where the server-side time is spent during a request can be a challenging task when you are new to an existing codebase. If you’re trying to address the generic/non-specific “we need to improve performance / it’s slow” issue, you need to get a good picture of what is going on and where to prioritise effort.

Trust your developer instinct and experience

adathedev@gmail.com (Adrian Hills) — Fri, 13 Apr 2012 11:59:00 +0100

That feature you’re implementing, that bug you’re fixing, that performance improvement you’re trying to make…it’s using up a fair few brain CPU cycles trying to analyse how best to implement it. Scalability, performance, ease of maintenance, clean code…just some of the things you are mentally scoring your potential approaches against to determine which route to take. And that’s without any mention of business domain logic. We’ve all been in the situation where you start heading down one path only to find a hurdle.

Ordnance Survey Data Importer Coordinate Conversion Accuracy

adathedev@gmail.com (Adrian Hills) — Wed, 14 Mar 2012 22:11:00 +0000

Update 13 March 2013: Please see latest blog post (fix) on this here. Thanks to a comment on my original post around my project that imports Ordnance Survey CodePoint data into SQL Server, I was made aware of a potential issue with the (awesome) third party GeoCoordConversion DLL I use to convert the Eastings/Northings coordinates supplied in the Ordnance Survey data files, into Latitude/Longitude coordinates. The issue relates to an inaccuracy in the conversion process, specifically to do with integer divisions instead of double.

Quick win - check your table variable use

adathedev@gmail.com (Adrian Hills) — Mon, 05 Mar 2012 09:30:00 +0000

Quick wins are awesome. Making a change that takes minimal effort and yields a significant performance improvement is very satisfying. This particular potential quick win relates to the use of table variables vs. temporary tables. Have a non-trivial stored procedure that produces some intermediary results and stores in a table variable which then goes on to be used further in the stored procedure? Consider evaluating a switch to a temporary table instead.

Automating Web Performance Stats Collection in .NET

adathedev@gmail.com (Adrian Hills) — Mon, 20 Feb 2012 21:36:00 +0000

You have a web application. You’re a .NET developer. Maybe you already have some automated UI testing in place via Selenium, or maybe you don’t. What you want to do is automate the collection of some performance metrics about your application. Q. How would you go about doing that in .NET? A. Use the following recipe for success. Ingredients BrowserMob Proxy by Webmetrics, which (quote) is: A free utility to help web developers watch and manipulate network traffic from their web applications

Excluding nodes from XML data before returning from SQL Server

adathedev@gmail.com (Adrian Hills) — Mon, 16 Jan 2012 10:36:00 +0000

This post follows on from a question I recently replied to, for how to exclude a specific node from an XML column value before returning it, using TSQL. Example setup CREATE TABLE Example ( ID INTEGER IDENTITY(1,1) PRIMARY KEY, XmlField XML ) INSERT Example (XmlField) VALUES ('<Root><ChildA>Value I want to see</ChildA><ChildB>Value I do not want to see</ChildB></Root>') So if you want to return the XML minus the ChildB node, how do you do it?

Round up of 2011

adathedev@gmail.com (Adrian Hills) — Mon, 02 Jan 2012 22:32:00 +0000

Another year is done and dusted so it’s that time again - time to reflect on the past year. Quick Timeline March: Mongo UK - London A chance to attend my first NOSQL conference, having recently entered in the world of MongoDB for some projects. Picked up some good pointers and food-for-thought. Just some words: NOSQL, hard seats, gonna-need-a-bigger-venue, enlightening. April: SQLBits 8 - Beside The Seaside - Brighton Highlight of the year for me, conference-wise (was unable to make SQLBits 9 - Liverpool).

OS CodePoint Data Geography Update Sept 2011

adathedev@gmail.com (Adrian Hills) — Thu, 15 Sep 2011 21:44:00 +0100

History: GB Post Code Geographic Data Load to SQL Server using .NET OS CodePoint Data Geography Load Update Following a comment on my original post listed above, it appears the structure of the CodePoint data file has changed. This means that the importer will fail when attempting to import the latest data files supplied by Ordnance Survey as the columns in the files have changed. I’ve pushed an update to the project on GitHub to address this - it now takes a sensible approach so the index positions are no longer hard coded.

Getting Started with Riak and .NET

adathedev@gmail.com (Adrian Hills) — Thu, 18 Aug 2011 21:44:00 +0100

Earlier in the year, I started playing around with MongoDB using .NET and wrote up a Getting Started guide. Now it’s Riak’s turn. Nutshell Riak is a distributed, schemaless, data-type agnostic key-value “NoSQL” database written primarily in Erlang. Check out the “What is Riak?” page on Basho’s wiki. Riak does not run on Windows so you’ll need to choose your supported OS of choice to install Riak on. Having used Ubuntu before for MongoDB, I went down that route so my notes here are oriented that way.

sp_executesql change between 2005 and 2008

adathedev@gmail.com (Adrian Hills) — Mon, 27 Jun 2011 20:28:00 +0100

Today I tripped over what turned out to be a difference in the way sp_executesql behaves between SQL Server 2005 and 2008 when executing a string containing a parameterised stored procedure call. Take this simplified example: DECLARE @SQL NVARCHAR(256) SET @SQL = 'sp_help @obj' EXECUTE sp_executesql @SQL, N'@obj NVARCHAR(100)', 'sp_help' In SQL Server 2008 (10.0.4000.0), the above executes successfully. In SQL Server 2005 (9.00.1399.06), it throws the following exception: Msg 102, Level 15, State 1, Line 1 Incorrect syntax near 'sp_help'.

The importance of "Working Set"

adathedev@gmail.com (Adrian Hills) — Fri, 24 Jun 2011 20:55:00 +0100

One of the things that I see cropping up pretty often is this thing called “working set”. After recently chipping in on another StackOverflow question on the subject of “What does it meant to fit ‘working set’ in RAM?”, I thought it was a good subject for a blog post. This is really just a copy and extension of my input on that question and focused in certain parts on MongoDB, but is also as relevant to other databases.

Upcoming SQLSoton UG - 6th July 2011

adathedev@gmail.com (Adrian Hills) — Thu, 23 Jun 2011 14:15:00 +0100

The next SQLSoton (twitter) UserGroup meet is coming up on Wednesday 6th July. Matt Whitfield (twitter) will be presenting some “notes from the field” and Alex Whittles (twitter | blog) will be giving a talk on automating cube documentation with SSRS, DMV & spatial data. Don’t forget to register! Oh, and did I forget to mention? There’s an XBox up for grabs. See you there? Thought so :)

If you can't make it, untake it

adathedev@gmail.com (Adrian Hills) — Wed, 22 Jun 2011 11:33:00 +0100

There’s some great free events out there; SQLBits community day and the upcoming SQL In The City day by Red Gate are just two examples. With the high demand for spaces at events like these, it is well worth getting in there ASAP to reserve your space before they all go and you end up on a waiting list. But if you find you can no longer make it, make someone else’s day by cancelling your place so that it can be freed up for those on the waiting list.

MongoDB ASP.NET Session State Store Provider

adathedev@gmail.com (Adrian Hills) — Tue, 03 May 2011 21:03:00 +0100

Please note: v1.1.0 of this Sesssion State Provider is now available. See blog post: https://www.adathedev.co.uk/2013/03/mongodb-aspnet-session-store-provider.html I’ve pushed up an initial drop of a custom ASP.NET session state store, backed by MongoDB, to my GitHub repository. After a quick test it seems OK, though could do with a real hammering before being deemed production worthy! So if you want to use it, just make sure you give it a good test through and most importantly, let me know how it goes!

Microsoft Community Contributor Award 2011

adathedev@gmail.com (Adrian Hills) — Fri, 29 Apr 2011 20:22:00 +0100

For a number of years now (since 2002 to be exact), I’ve been chipping in with my 2 pence on various online Q&A sites. As I mention in one of my first blog posts from just over a year ago, “Ways to improve technical skills” and one back in August on “The importance of being earnest”, participating in sites like StackOverflow and the MSDN forums is a great way to keep you on your toes technically.

5 Tips For Attending Conferences

adathedev@gmail.com (Adrian Hills) — Mon, 04 Apr 2011 12:01:00 +0100

With SQLBits 8 being this week in Brighton and it being set to be the biggest and best one yet, it got me thinking. If I was a first timer to a conference like this, what should I expect? So here’s my top 5 tips: Talk. It’s easy to be daunted by a room full of people who all seem to be in little groups chatting away. You think everyone else there knows someone, except you.

OS Data Importer Supports Scale Gazetteer Dataset

adathedev@gmail.com (Adrian Hills) — Sun, 27 Mar 2011 21:54:00 +0100

I’ve just pushed some updates to GitHub for the Ordnance Survey Data Importer .NET app I’ve been working on every now and then (see my previous posts: GB Post Code Geographic Data Load to SQL Server using .NET and OS CodePoint Data Geography Load Update). Aside from a little re-jigging to potentially pave the way to load more Ordnance Survey data files to SQL Server in the future, it now supports importing the 1:50000 Scale Gazetteer data file to SQL Server (available to download from here).

MongoDB Journaling Performance - Single Server Durability

adathedev@gmail.com (Adrian Hills) — Fri, 18 Mar 2011 10:34:00 +0000

Note: Relates to v1.8.0 release. v1.8 of MongoDB (production release hot off the press, available from here) brings single server durability to MongoDB via its support for write-ahead journaling. The following summarises the key points from the journaling documentation, administration guide and from my initial playing around. you have to specify the –journal switch for mongod in order to enable journaling (was previously –dur prior to production release) journal files are created within a journal subfolder of your dbpath, up to 1GB each in size in the event of a crash, when mongod restarts it will replay the journal files to recover automatically before the server goes online if mongod shuts down cleanly, the journal files are cleared down.

Upcoming DevConnections London conference

adathedev@gmail.com (Adrian Hills) — Thu, 17 Mar 2011 08:55:00 +0000

What: IT&DevCONNECTIONS Conference Where: ExCel Conference Centre, London When: 13th - 15th June 2011 Here’s the summary on the event, which gives a good overview of what you can expect from it. Hold on to your hat for some very impressive name-dropping! Penton, publishers of SharePointPro Connections, DevPro Connections, Windows IT Pro and SQL Server Magazines are excited to announce the IT&DevCONNECTIONS powered by Microsoft conference to be held at the ExCeL conference centre in London.

SQLSoton UG March 2011

adathedev@gmail.com (Adrian Hills) — Thu, 10 Mar 2011 22:13:00 +0000

Last night saw another gathering of the SQL Server Southampton UserGroup, with Gavin Payne (Twitter | Blog) making the trip down to give a talk on replication, his first since joining Coeo. As a developer, I’ve had very little exposure to replication and have seen it as some form of dark art. I’ve never scratched much past the surface. Over the past couple of months of getting up to speed with MongoDB, I’ve dived far deeper into replication within that than I ever have in SQL Server in 10 years+ that I’ve been using it which is quite a strange feeling, thinking about it like that.

MongoDB 1.8.0 Release Candidate 2 Available

adathedev@gmail.com (Adrian Hills) — Thu, 10 Mar 2011 10:04:00 +0000

Fast on the heels of my previous blog post about MongoDB 1.8.0 Release Candidate 1 becoming available, I’ve just seen 1.8.0 RC2 is now available which includes a handful of changes (Download | Change Log). Did I mention, MongoDB is rapidly evolving?!

Upcoming SQLSoton UG - 9th March 2011

adathedev@gmail.com (Adrian Hills) — Thu, 03 Mar 2011 10:33:00 +0000

Quick reminder that the next SQLSoton (twitter) UserGroup meet is fast approaching on Wednesday 9th March. Gavin Payne (twitter | blog) will be making his way down to Southampton to give us a talk on replication: For those who have never had the chance to use SQL Server’s replication features, or perhaps only manage an existing implementation of it, this session will give you an overview of the different replication services SQL Server offers, some background into the primary and secondary design goals each, along with some considerations for deployment and in-life management.

Thoughts on MongoDB from a SQL Server Dev

adathedev@gmail.com (Adrian Hills) — Fri, 25 Feb 2011 14:20:00 +0000

As someone with a background in SQL Server development dating back to 2000, the whole NoSQL jazz has been something that’s been on the radar for a while but never made it under the microscope so to speak. Partly because SQL Server is my comfort zone and there’s plenty more to still learn about that; partly because I’ve had my career mapped out in my head and NoSQL didn’t feature much in that vision; partly because until you have something real-world driving a push into a technology then you tend to have that technology remaining as a distant dot on the radar.

MongoDB 1.8.0 Release Candidate is out

adathedev@gmail.com (Adrian Hills) — Thu, 24 Feb 2011 09:26:00 +0000

MongoDB 1.8.0-rc0 is now available, you can download from here and find the release notes here. The main changes over the current 1.6 production release relate to: durability, a much talked about topic (!), with support now for write-ahead journaling (docs) sparse and covered indexes (on my radar to try out) support for more Map/Reduce functionality (note there are some breaking changes in this area) The full info on the changes is in the Release Notes document.

MongoUK 2011 - Upcoming MongoDB event

adathedev@gmail.com (Adrian Hills) — Wed, 23 Feb 2011 15:45:00 +0000

There’s a one day conference on MongoDB coming up, being held at the Skills Matter eXchange in London on Monday 21st March 2011. If you’re interested in MongoDB or even just curious as to what it’s all about, then it’s worth registering for it based on what’s on the agenda - it’s going to be tough to choose between the different sessions, which include ones on: schema design storage engine internals indexing and query optimiser scalability map/reduce & geo indexing It will also be a good chance to hear from speakers outside of 10gen such as Thoughtworks, Guardian and Boxed Ice.

SQLSoton UserGroup Room Change from March

adathedev@gmail.com (Adrian Hills) — Mon, 21 Feb 2011 20:34:00 +0000

Due to the success of the SQL Southampton User Group and based on the feedback/opinions of those who have attended, the room the group is held in is changing. It’s still in the same venue, Avenue St Andrews United Reformed Church, but instead of being in the Upper Room as per the last few meets, it will be in St Andrew’s Hall. This gives a bigger, more flexible room. The now famous #SQLSoton signs will be in place to direct as per usual - still round via the back carpark but a different entrance now.

MongoDB replication - oplogSize

adathedev@gmail.com (Adrian Hills) — Mon, 21 Feb 2011 11:10:00 +0000

Background An oplog is a write operation log that MongoDB uses to store data modifications that need to be replicated out to other nodes in the configured set. This oplog is a capped collection which means it will never grow in size beyond a certain point - once it reaches it’s max size, old data drops off the end as new data is added so it keeps cycling round. The size of the oplog basically determines how long a secondary node can be down for and still be able to catch up when it comes back online.

OS CodePoint Data Geography Load Update

adathedev@gmail.com (Adrian Hills) — Sun, 13 Feb 2011 19:07:00 +0000

Following on from my previous post on loading the Ordnance Survey Code-Point data for GB post codes to SQL Server and converting to the GEOGRAPHY data type, I’ve made a few tweaks to the importer app that is up on GitHub: The schema of the SQL Server table generated has changed - postcodes are now split into 2 distinct columns: OutwardCode and InwardCode The importer now calculates a basic average for each postcode district (e.

MongoDB - Does My Data Look Big In This?

adathedev@gmail.com (Adrian Hills) — Mon, 07 Feb 2011 10:10:00 +0000

You have an existing relational database containing x amount of data and you decide to migrate that data, for whatever reason, into MongoDB. You may have an pre-conceived belief that as your relational database is x GB in size, that after loading that data into MongoDB your NoSQL database size will be around about x GB too - after all, you’re loading exactly the same data from one to the other right?

GB Post Code Geographic Data Load to SQL Server using .NET

adathedev@gmail.com (Adrian Hills) — Mon, 24 Jan 2011 08:38:00 +0000

Ordnance Survey now make available a number of mapping data/geographic datasets to download for free, allowing unrestricted use for commercial and non-commercial use. One of these is the Code-Point Open dataset, which gives a precise geographic location of each post code in Great Britain as a CSV file. This file contains the Easting and Northing coordinates for each postcode. You may want to convert these to Latitude/Longitude coordinates to then load into a GEOGRAPHY column in SQL Server (as of 2008) and do all kinds of spatial wizardry in the database.

Running MongoDB as a Windows Service

adathedev@gmail.com (Adrian Hills) — Tue, 18 Jan 2011 09:58:00 +0000

Following on from my opening “Getting started with MongoDB and .NET” post, one of the next logical steps was to get mongodb running as a Windows service. From a command prompt, in the mongodb bin directory run: mongod --install --serviceName "MongoDB" --dbpath C:\mongodb\data\db --logpath C:\mongodb\logs\mongolog.txt --logappend Obviously you can tweak the arguments for your environment - I opted to use a different dbpath to the default \data\db Starting the service is then just a case of running:

SqlBulkCopy to SQL Server in Parallel

adathedev@gmail.com (Adrian Hills) — Wed, 12 Jan 2011 10:32:00 +0000

In an earlier post last year, I blogged about high performance bulk loading to SQL Server from .NET using SqlBulkCopy. That post highlighted the performance gain that SqlBulkCopy gives over another batched insert approach using an SqlDataAdapter. But is it possible to squeeze more performance out? Oh yes. First, a quick recap. For optimal performance: load into a heap table (with no indexes - add any indexes you need AFTER you’ve loaded the data)

GROUPING SETS in SQL Server

adathedev@gmail.com (Adrian Hills) — Mon, 10 Jan 2011 12:09:00 +0000

Something I personally haven’t seen a lot of out there in the SQL Server world, is use of GROUPING SETS - an operator that can be applied in a GROUP BY clause. So what does it do? How would you use it? Take the AdventureWorks sample database as an example playground. Suppose you want to query the sales data to find the following: total sales for each product total sales for each product category total sales There’s a number of ways you could do this.

.NET Admin GUI for MongoDB

adathedev@gmail.com (Adrian Hills) — Mon, 10 Jan 2011 09:04:00 +0000

Having just started out fiddling around with MongoDB from .NET, finding a GUI over the top of MongoDB to help me out while I find my feet was one of the first things I looked for. And I’ve been using the first tool I found : MongoVUE - there undoubtedly are others out there, but at the moment I’ve found no reason to continue searching. Can it do everything I want it to?

Getting Started with MongoDB and .NET

adathedev@gmail.com (Adrian Hills) — Fri, 07 Jan 2011 21:28:00 +0000

Getting up and running with MongoDB using C# is a very simple process. It literally takes a couple of minutes, including the time to download MongoDB itself and the C# drivers - so if you fancy having a play around with it to see what it’s all about, there’s no barrier stopping you. Nutshell To quote MongoDB.org: MongoDB (from “humongous”) is a scalable, high-performance, open source, document-oriented database. Check out the MongoDB homepage for a highlight of the key features.

Round up of 2010

adathedev@gmail.com (Adrian Hills) — Thu, 30 Dec 2010 21:36:00 +0000

2010 was my first year as a blogger, after finally pulling my finger out back in February. I learnt an early lesson in naming/tagging posts appropriately after my first search engine hits came in via the term “pull my finger”. Not exactly the kind of search term I was hoping to bring up my site - things have improved now so I’ve headed in the right direction with a focus on SQL Server and .

SqlBits 8 - Upcoming SQL Server event

adathedev@gmail.com (Adrian Hills) — Thu, 23 Dec 2010 12:03:00 +0000

So SQLBits 8 has been announced and is going to be beside the seaside (so maybe it should be called SeaQLBits) in that there Brighton at The Grand, from Thursday 7th April to Saturday 9th April 2011. If you’re reading my blog, then there’s a pretty strong chance you use SQL Server in which case without a doubt, SQLBits will have a lot to offer you. Chalk it down in your calendar.

SQLSoton UserGroup Jan 2011

adathedev@gmail.com (Adrian Hills) — Sun, 19 Dec 2010 21:03:00 +0000

After the success of the first ever SQL Server Southampton UserGroup this month, the next is planned for Wednesday 19th January 2011 with Christian Bolton (Blog| Twitter) doing a talk on “Exploring SQL Server’s Architecture”. Followed by SQL Supper - a chance to eat pizza and discuss anything SQL Server related. Got a niggle at work you’d like 2nd/3rd/nth opinions on? Perfect opportunity. Just want to network and have a casual chat?

SQL Server Southampton UserGroup

adathedev@gmail.com (Adrian Hills) — Thu, 09 Dec 2010 08:15:00 +0000

Last night was the first SQL Server Southampton UserGroup (#sqlsoton for those on twitter) organised by Mark Pryce-Maher (Twitter | Blog). It’s a difficult job trying to get a local usergroup up and running especially when it’s difficult to reach out to local professionals and get a firm idea of who’s going to come, and so credit to Mark for making it happen. Cap’n, we have no power Despite projector/power lead problems (and there being a million and one kettles in the hired room, none with suitable lead!

SQLBits 6 videos available

adathedev@gmail.com (Adrian Hills) — Thu, 21 Oct 2010 08:20:00 +0100

First tweet I say today was a great one - the SQLBits 6 videos are now available. Good news from Simon Sabin (Blog | Twitter) - thanks once again, and to the whole SQLBits team. If you were there, then no doubt you’re like me and keen to either catch the sessions you didn’t attend or re-watch some sessions you did attend. If you weren’t there, then even more reason to check them out (list of sessions).

SSMS Tools Pack

adathedev@gmail.com (Adrian Hills) — Wed, 20 Oct 2010 20:40:00 +0100

Today I downloaded and installed the SSMS Tools Pack, a free add-in for SQL Server Management Studio developed by Mladen Prajdić (Blog | Twitter). Late to the party…and not even fashionably late Yes I know, I know - everyone uses the SSMS Tools Pack and I’m well aware I’m late to the party! Fail whale on my part. It’s one of those things I bookmarked a long time ago and just never got round to actually trying out.

Long time no post

adathedev@gmail.com (Adrian Hills) — Mon, 18 Oct 2010 17:14:00 +0100

It’s been a while since my last blog post, with good reason as September saw the arrival of my first child - a beautiful baby girl. It did mean I had to miss SQLBits 7 up in York, but of course that pales into insignificance when comparing to becoming a Dad for the first time! Besides, there’s always SQLBits 8…(cough bring it to the south coast! cough). Now I’m back at work after paternity leave and I’m starting to get back in the swing of things.

Passing a TABLE variable into dynamic SQL

adathedev@gmail.com (Adrian Hills) — Tue, 10 Aug 2010 19:55:00 +0100

A question popped up on StackOverflow today, asking how to pass a TABLE variable into a dynamic SQL statement in SQL Server 2008. I’ve previously blogged about table-valued parameters, comparing the approach of passing in a TABLE of values to a stored procedure to the techniques you’d have had to use in earlier versions of SQL Server (e.g. CSV or XML), but this specific question is worth a quick follow-up.

The importance of being earnest

adathedev@gmail.com (Adrian Hills) — Thu, 05 Aug 2010 23:40:00 +0100

I’ve been learning and trying out a lot of new things recently and it got me to thinking just how important it is in this business, to continue pushing yourself and adding more strings to your bow. It can be easy to stay within your comfort zone and go with the flow, but there (usually) comes a time when you have to branch out for one reason or another. Stand-ups, Tomatoes and a date(-abase) with Cassandra Over the past week we’ve started doing daily stand-ups at work - still in the early stages but it already feels like I’ve been doing it for longer.

Getting started with Cassandra and .NET

adathedev@gmail.com (Adrian Hills) — Mon, 05 Jul 2010 23:05:00 +0100

Over the past couple of days, I’ve started playing around with Cassandra in an effort to satisfy my curiosity and to see what it’s all about. In terms of databases, SQL Server is my main skill - I first started using it nearly 10 years ago, and will always remember the first book I read cover to cover on it: Inside Microsoft SQL Server 6.5 by Ron Soukup. So Cassandra is a step outside of my comfort zone; something new and a little alien to me….

Validating an SQL query programmatically, with gotcha

adathedev@gmail.com (Adrian Hills) — Thu, 01 Jul 2010 22:31:00 +0100

If you want to check the validity of a TSQL statement programmatically, you can make use of the SET NOEXEC statement. As this MSDN reference states, SET NOEXEC ON will compile the query but won’t actually execute it. This is ideal (well, nearly….) if perhaps you have a dynamically generated statement that you want to check is valid before executing it. Example time SET NOEXEC ON GO SELECT TOP * FROM sys.

The SQL Server MasterClass Experience

adathedev@gmail.com (Adrian Hills) — Fri, 18 Jun 2010 22:18:00 +0100

SQL Server MasterClass 2010 was a very high quality, one day event presented by Paul S. Randal (Twitter | Blog) and Kimberly L. Tripp (Twitter | Blog) - not that they need any introduction of course. In fact if you don’t know of them, then perhaps you should close down your SSMS window and switch off your machine - after reading my blog of course :). Maybe uninstall SQL Server too…

SQL Server XML datatype with CDATA

adathedev@gmail.com (Adrian Hills) — Mon, 14 Jun 2010 14:06:00 +0100

So today I learnt something new - it turns out the XML datatype in SQL Server does not preserve CDATA sections. e.g. DECLARE @XML XML SET @XML = '<Test><NodeA><![CDATA[Testing cdata section <woop!>]]></NodeA></Test>' SELECT @XML Results ------------------------------------------------------------------ <Test><NodeA>Testing cdata section <woop!></NodeA></Test> After a quick dig around, I found this MS Connect case. I personally would like to see it accept whatever you pass in without silently altering it, as long as it’s well-formed XML of course.

Optimising wildcard prefixed LIKE conditions

adathedev@gmail.com (Adrian Hills) — Wed, 26 May 2010 08:41:00 +0100

Suppose you want to write a query to find all products in your database that have a name beginning with “Long-Sleeve”. You’d more than likely use something like below (examples based on AdventureWorks LT sample database): SELECT name FROM SalesLT.Product WHERE name LIKE 'Long-Sleeve%' This produces a good execution plan, performing an index seek on the nonclustered index that exists on the name column. If you look at the seek operation, you’ll see the query optimiser has done a good job of ensuring an index can be used by looking at the seek predicate:

QCon London 2010 Videos

adathedev@gmail.com (Adrian Hills) — Mon, 24 May 2010 22:00:00 +0100

It’s been a couple of months now since QCon London (check out my previous blog post on The QCon London 2010 Experience) and some session videos are starting to make their way out on InfoQ. With a lot of great content, they’re well worth checking out. I’ve re-watched the sessions I attended and enjoyed them as much the second time round as I did the first time. The State of the Art on .

The SQLBits VI Experience

adathedev@gmail.com (Adrian Hills) — Mon, 19 Apr 2010 08:20:00 +0100

I had high expectations about SQLBits VI and it certainly lived up to it! Not only did it have great speakers, who really know their stuff and have an effortless way of getting the information across, but it was also free. So it goes without saying that it was a popular event. There were 4 sessions going on at any one time with a theme of performance and scalability and plenty of choice across DBA, Dev and BI areas.

SqlBulkCopy ColumnMappings Mismatch

adathedev@gmail.com (Adrian Hills) — Wed, 14 Apr 2010 22:49:00 +0100

“The given ColumnMapping does not match up with any column in the source or destination” This error caused me more head-scratching than it should have done today. I was using SqlBulkCopy to bulk insert some data into an SQL Server table from a .NET DataTable, something I’ve done before - it’s not rocket science. But I hit this error upon calling WriteToServer and it took me a while (longer than it should have done really!

SQL CAST to VARCHAR without size

adathedev@gmail.com (Adrian Hills) — Tue, 06 Apr 2010 19:27:00 +0100

What’s the result of this, in SQL Server? SELECT CAST(12345 AS VARCHAR) As you expect, it’s “12345”. How about this? SELECT CAST(1234567890123456789012345678901 AS VARCHAR) You may be expecting it to return “1234567890123456789012345678901” but it will in fact throw an error: Msg 8115, Level 16, State 5, Line 1 Arithmetic overflow error converting numeric to data type varchar. It’s all because a length was not given for the VARCHAR to convert the value to.

The QCon London 2010 Experience

adathedev@gmail.com (Adrian Hills) — Sun, 14 Mar 2010 15:59:00 +0000

QCon London 2010 - what a conference! Awesome…with a capital AWE So many inspiring speakers, a packed schedule of talks over 3 days on a number of tracks and a great bunch of people. This was my first QCon, and cannot recommend the experience enough. The only thing I didn’t enjoy, was having to decide between which talks to attend. But that’s the true sign of a good conference. I’m finding it difficult to write a blog post as my mind is still buzzing and there was a lot to soak up.

Rise of the SQL Server DevBA

adathedev@gmail.com (Adrian Hills) — Thu, 04 Mar 2010 21:30:00 +0000

Something that has got me thinking recently is the distinction between an SQL Server Developer and a DBA. I imagine that most people would describe themselves as one or the other exclusively. My CV for example says I’m a developer - that is what I am. I would never market myself as a DBA; aside from the fact I just don’t have the full skillset of a DBA, it would just be an insult to the real DBAs out there whose knowledge and experience in that arena far outweighs mine.

Queue table processing in SQL Server

adathedev@gmail.com (Adrian Hills) — Mon, 01 Mar 2010 22:27:00 +0000

Implementing SQL Server queue table processing logic is something I keep meaning to blog about and finally I’ve got round to it thanks to my memory being jogged by StackOverflow questions I’ve recently participated in, including this one. The scenario is you queue up records in a database table, each representing a piece of work needing to be done. You then want to have processes that periodically poll this table to pick up the next item of work from the queue and process them.

Changing primary key index structure

adathedev@gmail.com (Adrian Hills) — Mon, 01 Mar 2010 08:00:00 +0000

Changing the structure of a primary key constraint index from nonclustered to clustered (or from clustered to nonclustered) is not necessarily as straight forward as it first seems. The process of changing it over, involves the constraint being dropped and then recreated. This could potentially cause a problem if you’re making the change on a table whilst there could be activity against it. Example TableX was originally created as below:

SQLBits VI - Upcoming SQL Server event

adathedev@gmail.com (Adrian Hills) — Thu, 25 Feb 2010 20:32:00 +0000

SQLBits VI is scheduled in my calendar for Friday 16th April - and I’m expecting great things! It’s a 1 day, SQL Server event being held in London (Westminster to be semi-precise), and the theme is performance and scalability which, as I mentioned in an earlier blog post, is particularly high on my agenda at the moment. Not only does it offer high quality speakers, but it’s a free event too.

Autogenerated SQL constraint names - why to avoid them

adathedev@gmail.com (Adrian Hills) — Tue, 23 Feb 2010 21:42:00 +0000

Autogenerated constraint names in SQL Server - are they convenient, or are they a hindrance? For me, they are a hindrance. Why should I care that SQL Server autogenerates a constraint name when I don’t explicitly specify one? Why does it matter if it names my PRIMARY KEY constraint “PK__Constrai__3214EC271FCDBCEB”, or my DEFAULT constraint “DF__Constraint__ColumnA__21B6055D”? Why should I name each constraint myself and not rely on autogenerated constraint names? Because it makes maintenance and database update deployments easier, simpler and in my opinion, safer.

Optimising date filtered SQL queries

adathedev@gmail.com (Adrian Hills) — Sun, 21 Feb 2010 16:55:00 +0000

How you structure your SQL queries is very important and choosing the wrong approach can have big effects on the performance of the query. One of the key things that should flag up the potential for needing optimisation, is if you are using a function/calculation within a WHERE clause. These can lead to inefficient execution plans, preventing sub-optimal index use. A classic example to demonstrate this is when querying the data for a specific month.

Using the SP:StmtCompleted SQL Profiler trace event class

adathedev@gmail.com (Adrian Hills) — Wed, 17 Feb 2010 22:42:00 +0000

One question I’ve seen popping up a few times recently, is how to check what individual statement(s) within an SQL Server stored procedure are taking the most time. The scenario is that there is a stored procedure that consists of a number of SQL statements. The stored procedure has been flagged up as running slower than perhaps it was expected to, but as there are a number of statements/queries within it, it’s not immediately clear where the time is being taken.

High performance bulk loading to SQL Server using SqlBulkCopy

adathedev@gmail.com (Adrian Hills) — Mon, 15 Feb 2010 21:24:00 +0000

If you ever want to bulk load data into an SQL Server database as quickly as possible, the SqlBulkCopy class is your friend (in the System.Data.SqlClient namespace). Since being introduced in .NET 2.0, it has provided an extremely efficient way to bulk load data into SQL Server, and is one the classes that I see as a “must know about”. A usual scenario is where you want to dump some data into the database to then do some processing on.

Ways to improve technical skills

adathedev@gmail.com (Adrian Hills) — Sat, 13 Feb 2010 12:41:00 +0000

It’s nearly the end of my first week as a blogger. Already I’m feeling the benefits of finally getting round to starting up my blog. It got me thinking about what the best ways are to continue developing your skills and learning new things. For me, there are 3 particular ways that I find are invaluable if you want to keep yourself on your toes technically: Learn by doing - often you learn my making mistakes.

Sorting a DataTable - LINQ performance

adathedev@gmail.com (Adrian Hills) — Fri, 12 Feb 2010 21:00:00 +0000

Whenever there are a number of ways to achieve the same goal, I’m always inquisitive as what the performance difference is between them. I’m a firm believer in thinking about scalability from the start - if you can make your best effort to prepare for scale, then you can save yourself time and effort further down the line. I like to try and avoid pain - it doesn’t agree with me.

SQL Server 2008 - Table Valued Parameters vs XML vs CSV

adathedev@gmail.com (Adrian Hills) — Thu, 11 Feb 2010 20:38:00 +0000

The scenario is, you want to create a stored procedure that returns the records relating to a finite set of keys/IDs that you pass in to it. How do you do it? Well you could use dynamic SQL, but for the purpose of this article I’m going to ignore that option. Test scenario -- Create basic customer table CREATE TABLE [dbo].[Customer] ( Id INTEGER IDENTITY(1,1) PRIMARY KEY, Name NVARCHAR(50) NOT NULL ); GO -- Populate Customer table with 100,000 sample records DECLARE @Counter INTEGER SET @Counter = 1 WHILE (@Counter <= 100000) BEGIN INSERT Customer (Name) VALUES ('Test Customer #' + CAST(@Counter AS VARCHAR(10))) SET @Counter = @Counter + 1 END Option 1 - CSV list of keys In SQL Server 2000, you would most likely have ended up passing in a CSV list of keys, splitting that out into a table, and then joining that on to your real data table to match the records to return.

QCon London 2010

adathedev@gmail.com (Adrian Hills) — Wed, 10 Feb 2010 21:00:00 +0000

So QCon London is fast approaching, starting with 2 days of tutorials on the 8th and 9th of March, then the 3 day conference from 10th-12th. This will be my first time at QCon and I’m really looking forward to it. Having never been to many conferences in the past, the level of expectation I have is solely based on the StackOverflow DevDay in London last October. That was a great day with some very interesting (and humorous) talks, so I’m expecting even better things from QCon as it’s reputation precedes it!

Would you like SQL cache with that?

adathedev@gmail.com (Adrian Hills) — Tue, 09 Feb 2010 19:32:00 +0000

One of the things I feel I keep badgering on about in the world of SQL Server and query tuning is to be fair and consistent when comparing the different possible variants. If you don’t level the playing field, then potentially it will lead to you thinking a particular query out-performs another when in fact the opposite could be true. It’s a bit like comparing 2 paper boys to see which one can complete a given round the quickest - if one of them has been doing the round for the past 2 weeks but the other has never done that round before, then are you going to be able to reliably tell which one is actually quickest?

Pulling my finger out

adathedev@gmail.com (Adrian Hills) — Tue, 09 Feb 2010 13:40:00 +0000

Starting up a blog is something I’ve been threatening to do for some time now. There’s always a good excuse not to get round to it, even though it’s something I’ve really wanted to dip my toe into, just to see if I could cut the mustard. As it’s the start of what I hope will be a successful foray into blogging, I’m afraid this is indeed a “what do you do and where do you come from?

adathedev@gmail.com (Adrian Hills) — Mon, 01 Jan 0001 00:00:00 +0000

Full-stack C#, Azure, database developer (NOSQL & SQL). Remote worker. Always learning. Based in Hampshire, UK.