andyfrench.info

The problem

We have a system that has a number of separate Windows services that all access a shared database hosted by SQL Server 2008 R2. The system is used to provide online hydraulic modelling in support of a water utility company’s maintenance activities. The services are built around SynerGEE Water, a hydraulic modelling product from DNVGL.

One of the Windows services is responsible for monitoring a model library – SynerGEE models are stored as MDB files on the file system – and when a new or updated model is detected it adds a MODEL record to the database. It also indexes all of the pipes in the model and adds them to a MODEL_PIPE table in the database.

A second service checks the database for new MODEL records and then invokes SynerGEE to perform some hydraulic analysis. The results of this analysis are used to update the MODEL_PIPE records.

We observed that if a number of models had been updated in one go the result was that sometimes a database deadlock occurred when the second service was querying the MODEL_PIPE table. This was because the first service was in the process of adding other MODEL_PIPE records for other models at the same time.

We are using NHibernate for all data access and all database queries or updates are wrapped in transactions with the assistance of the Spring.Net transaction template. NHibernate Profiler was used to confirm that all the transactions were correctly formed and we could see that the transactions were using the READ COMMITTED isolation level.

The solution

Firstly, I did some research around Minimizing Deadlocks and noted that using a row-based isolation level can help. In particular, activating READ_COMMITTED_SNAPSHOT on the database can help by allowing SQL Server to use row versioning rather than shared locks.

“When the READ_COMMITTED_SNAPSHOT database option is set ON, a transaction running under read committed isolation level uses row versioning rather than shared locks during read operations.” [1]

Further research around snapshot isolation levels provided further insight:

“The READ_COMMITTED_SNAPSHOT database option determines the behavior of the default READ COMMITTED isolation level when snapshot isolation is enabled in a database. If you do not explicitly specify READ_COMMITTED_SNAPSHOT ON, READ COMMITTED is applied to all implicit transactions. This produces the same behavior as setting READ_COMMITTED_SNAPSHOT OFF (the default). When READ_COMMITTED_SNAPSHOT OFF is in effect, the Database Engine uses shared locks to enforce the default isolation level. If you set the READ_COMMITTED_SNAPSHOT database option to ON, the database engine uses row versioning and snapshot isolation as the default, instead of using locks to protect the data.” [2]

Bingo! If the READ_COMMITTED_SNAPSHOT database option is set to ON row versioning is used instead of locks.

“Once snapshot isolation is enabled, updated row versions for each transaction are maintained in tempdb. A unique transaction sequence number identifies each transaction, and these unique numbers are recorded for each row version. The transaction works with the most recent row versions having a sequence number before the sequence number of the transaction. Newer row versions created after the transaction has begun are ignored by the transaction…

…Snapshot isolation uses an optimistic concurrency model. If a snapshot transaction attempts to commit modifications to data that has changed since the transaction began, the transaction will roll back and an error will be raised. ” [2]

In our case this looked very interesting because the first Windows service adds new MODEL and MODEL_PIPE records in a transaction and is done with them. The second Windows service then reads the new MODEL and MODEL_PIPE records and updates them in a separate transaction. The chances of an optimistic concurrency issue are minimal. Although the two services are accessing the same table they are not accessing the same rows. Therefore, using row version-based locks would allow the two services to work better together.

So, I enabled READ_COMMITTED_SNAPSHOT on the database [3] and found that the deadlocks no longer occurred.

ALTER DATABASE <dbname> SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
ALTER DATABASE <dbname> SET READ_COMMITTED_SNAPSHOT ON;
ALTER DATABASE <dbname> SET MULTI_USER;

Result!

References

[1] Minimizing Deadlocks, Technet.

[2] Snapshot Isolation in SQL Server, MSDN.

[3] Enabling Row Versioning-Based Isolation Levels, Technet.

Read More

Every now and then I like to go back and have a look at something I think I know and double check my assumptions and understanding. In this case I’m going to look at the GROUP BY clause in SQL.

Partitions

Firstly, let’s look at partitions in sets. This is important because it directly relates to the ‘groups’ returned by a GROUP BY clause.

“A partition of a set X is a set of nonempty subsets of X such that every element x in X is in exactly one of these subsets.” *

We can derive certain properties of partitions:

• The union of all the partitions (subsets) returns the original set
• The intersection of the partitions (subsets) is empty

We can think of this like dividing a pizza into pieces. Each piece is a partition, and joining the partitions together gives us the original pizza.

The ‘groups’ returned by a GROUP BY clause are effectively simple partitions of the original set of data.

The GROUP BY clause

When we use a GROUP BY clause we are taking the resulting set of data from a query (consisting of a FROM and WHERE clauses) and then put the rows into groups (partitions) based on the values of the columns specified in the GROUP BY clause.

Each group becomes a single row in the result table. Each column in the row must be a characteristic of the group not of a single row in the group. Therefore the SELECT list must be made up of grouping columns or optional aggregate functions. Note also that groups, by definition, must have at least one row (i.e. they can’t be empty). his means that the result of a COUNT will never return zero when used in a query against a non-empty table. Groups are also distinct.

The resulting table of a GROUP BY is called a group table. All subsequent operations are executed on the rows in the group table rather than the original rows.

NULL values are generally treated as a single group.

The columns in the SELECT statement can be a subset of the columns in the GROUP BY clause, but the columns in the GROUP BY clause can never be a subset of the columns in the SELECT statement.

References

* Partition of a set, Wikipedia

Read More

The GO Command

The batch terminator. The GO command is not part of the Transact-SQL language but is used by the SQLCMD, OSQL and ISQL utilities that can also be used within Query Analyzer and the Query Editor window.

A batch is a set of T-SQL statements that are submitted for execution as a group. A script is a file containing set of T-SQL statements. One script can contain many batches.

Note: if you declare variables in a script and issue a GO command the variables will ne destroyed.

The GO command is sometimes required (e.g. when defining a stored procedure – the CREATE PROCEDURE statement must be the first statement in the query batch so any preceding statements must be followed by GO).

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_NULLS ON
GO

CREATE PROCEDURE dbo.MyProcedure
AS
BEGIN
 SELECT *
 FROM INFORMATION_SCHEMA.TABLES
END

SET QUOTED_IDENTIFIER OFF
GO

The Semicolon

The semicolon character is a statement terminator.

It is a part of the ANSI SQL-92 standard but was never used within Transact-SQL.

There are two situations in which you must use the semicolon:

• Where you use a Common Table Expression (CTE), and the CTE is not the first statement in the batch.
• Where you issue a Service Broker statement and the Service Broker statement is not the first statement in the batch.

See also: http://www.sqlservercentral.com/articles/SQL+Puzzles/thegocommandandthesemicolonterminator/2200/

Read More

Common Table Expressions (CTEs) were introduced in SQL Server 2005. They are view/derived tale-like construct.

“A common table expression (CTE) can be thought of as a temporary result set that is defined within the execution scope of a single SELECT, INSERT, UPDATE, DELETE, or CREATE VIEW statement. A CTE is similar to a derived table in that it is not stored as an object and lasts only for the duration of the query. Unlike a derived table, a CTE can be self-referencing and can be referenced multiple times in the same query.” - http://msdn.microsoft.com/en-us/library/ms190766.aspx (MSDN SQL Server 2008 documentation)

CTEs offers the advantage of improved readability and ease in maintenance of complex queries.

The basic structure of a CTE is:

WITH expression_name [ ( column_name [,...n] ) ]
AS
( CTE_query_definition )

You can then query the CTE as you would a table/view.

Example:

USE AdventureWorks;
GO
-- Define the CTE
WITH Sales_CTE (SalesPersonID, NumberOfOrders, MaxDate)
AS
(
    SELECT SalesPersonID, COUNT(*), MAX(OrderDate)
    FROM Sales.SalesOrderHeader
    GROUP BY SalesPersonID
)

-- Query using the CTE
SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,
       E.ManagerID, OM.NumberOfOrders, OM.MaxDate
FROM HumanResources.Employee AS E
    JOIN Sales_CTE AS OS -- Join against CTE as if it were a table
    ON E.EmployeeID = OS.SalesPersonID
    LEFT OUTER JOIN Sales_CTE AS OM -- CTE can be referenced more than once
    ON E.ManagerID = OM.SalesPersonID
ORDER BY E.EmployeeID;
GO

You can define multiple CTEs after the WITH keyword by separating each CTE with a comma.

A CTE can reference itself and previously defined CTEs in the same WITH clause. Forward referencing is not allowed.

Lots more here: http://msdn.microsoft.com/en-us/library/ms175972.aspx

Read More

Try using the following function:

USE [DatabaseNameHere]
GO

CREATE FUNCTION dbo.Split
    (
      @String VARCHAR(8000),
      @Delimiter CHAR(1)
    )
RETURNS @temptable TABLE ( items VARCHAR(8000) )
AS BEGIN     
    DECLARE @idx INT     
    DECLARE @slice VARCHAR(8000)     
    
    SELECT  @idx = 1     
    IF LEN(@String) < 1
        OR @String IS NULL 
        RETURN     
    
    WHILE @idx != 0     
        BEGIN     
            SET @idx = CHARINDEX(@Delimiter, @String)     
            IF @idx != 0 
                SET @slice = LEFT(@String, @idx - 1)     
            ELSE 
                SET @slice = @String     
  
            IF ( LEN(@slice) > 0 ) 
                INSERT  INTO @temptable ( Items )
                VALUES  ( @slice )     

            SET @String = RIGHT(@String, LEN(@String) - @idx)     
            IF LEN(@String) = 0 
                BREAK     
        END 
    RETURN     
   END

Then execure a query like the following:

SELECT TOP 10 * FROM dbo.Split('Item1,Item2,Item3',',')

The code was taken from here: http://www.logiclabz.com/sql-server/split-function-in-sql-server-to-break-comma-separated-strings-into-table.aspx

Read More

To restore a database use the following SQL:

USE [tempdb]
ALTER DATABASE [DatabaseName]

SET SINGLE_USER WITH
ROLLBACK IMMEDIATE
GO

RESTORE DATABASE [DatabaseName]
FROM DISK = 'C:\BackupName.BAK'
WITH MOVE 'MDF_logical_name' TO 'C:\MDFName.mdf',
MOVE 'LDF_logical_name' TO 'C:\LDFName.ldf'
GO

ALTER DATABASE [DatabaseName] SET MULTI_USER
GO

Note that this puts the database into single user mode first (i.e. it kicks off other users). Don’t forget the USE [tempdb] statement or you may find the database still in use and the restore may fail. If you’re doing a straight restore you can get the logical filenames etc. from the backup file:

RESTORE FILELISTONLY FROM DISK = 'C:\BackupName.BAK'
GO

Read More

To measure query performance you can use the SQL Profiler to log to a table then run the following queries.

Most frequent queries

SELECT
    DISTINCT
        CAST(textdata AS varchar(5000)) AS textdata,
        COUNT(duration) AS Occurences,
        AVG(duration) AS AvgDuration,
        SUM(duration) AS TotalDuration
FROM    [trace table name here]
WHERE   LoginName = 'login name here'
        AND textdata NOT LIKE '--%'
        AND CAST(textdata AS VARCHAR(5000)) <> 'exec sp_reset_connection'
GROUP BY CAST(textdata AS VARCHAR(5000))
ORDER BY COUNT(duration) DESC

Most inefficient queries

SELECT DISTINCT
        CAST(textdata AS VARCHAR(5000)) AS textdata,
        COUNT(duration) AS Occurences,
        AVG(duration) AS AvgDuration,
        SUM(duration) AS TotalDuration
FROM    [trace table name here]
WHERE   LoginName = 'login name here'
        AND textdata NOT LIKE '--%'
        AND CAST(textdata AS VARCHAR(5000)) <> 'exec sp_reset_connection'
GROUP BY CAST(textdata AS VARCHAR(5000))
ORDER BY AVG(duration) DESC

Read More

To generate insert statements from existing data create and run the following stored procedure (taken from http://vyaskn.tripod.com/code.htm).

SET NOCOUNT ON
GO

PRINT 'Using Master database'
USE master
GO

PRINT 'Checking for the existence of this procedure'
IF (SELECT OBJECT_ID('sp_generate_inserts','P')) IS NOT NULL --means, the procedure already exists
 BEGIN
  PRINT 'Procedure already exists. So, dropping it'
  DROP PROC sp_generate_inserts
 END
GO

--Turn system object marking on
EXEC master.dbo.sp_MS_upd_sysobj_category 1
GO

CREATE PROC sp_generate_inserts
(
 @table_name varchar(776),    -- The table/view for which the INSERT statements will be generated using the existing data
 @target_table varchar(776) = NULL,  -- Use this parameter to specify a different table name into which the data will be inserted
 @include_column_list bit = 1,  -- Use this parameter to include/ommit column list in the generated INSERT statement
 @from varchar(800) = NULL,   -- Use this parameter to filter the rows based on a filter condition (using WHERE)
 @include_timestamp bit = 0,   -- Specify 1 for this parameter, if you want to include the TIMESTAMP/ROWVERSION column's data in the INSERT statement
 @debug_mode bit = 0,   -- If @debug_mode is set to 1, the SQL statements constructed by this procedure will be printed for later examination
 @owner varchar(64) = NULL,  -- Use this parameter if you are not the owner of the table
 @ommit_images bit = 0,   -- Use this parameter to generate INSERT statements by omitting the 'image' columns
 @ommit_identity bit = 0,  -- Use this parameter to ommit the identity columns
 @top int = NULL,   -- Use this parameter to generate INSERT statements only for the TOP n rows
 @cols_to_include varchar(8000) = NULL, -- List of columns to be included in the INSERT statement
 @cols_to_exclude varchar(8000) = NULL, -- List of columns to be excluded from the INSERT statement
 @disable_constraints bit = 0,  -- When 1, disables foreign key constraints and enables them after the INSERT statements
 @ommit_computed_cols bit = 0  -- When 1, computed columns will not be included in the INSERT statement
 
)
AS
BEGIN

/***********************************************************************************************************
Procedure: sp_generate_inserts  (Build 22) 
  (Copyright © 2002 Narayana Vyas Kondreddi. All rights reserved.)
                                          
Purpose: To generate INSERT statements from existing data. 
  These INSERTS can be executed to regenerate the data at some other location.
  This procedure is also useful to create a database setup, where in you can 
  script your data along with your table definitions.

Written by: Narayana Vyas Kondreddi
         http://vyaskn.tripod.com

Acknowledgements:
  Divya Kalra -- For beta testing
  Mark Charsley -- For reporting a problem with scripting uniqueidentifier columns with NULL values
  Artur Zeygman -- For helping me simplify a bit of code for handling non-dbo owned tables
  Joris Laperre   -- For reporting a regression bug in handling text/ntext columns

Tested on:  SQL Server 7.0 and SQL Server 2000

Date created: January 17th 2001 21:52 GMT

Date modified: May 1st 2002 19:50 GMT

Email:   vyaskn@hotmail.com

NOTE:  This procedure may not work with tables with too many columns.
  Results can be unpredictable with huge text columns or SQL Server 2000's sql_variant data types
  Whenever possible, Use @include_column_list parameter to ommit column list in the INSERT statement, for better results
  IMPORTANT: This procedure is not tested with internation data (Extended characters or Unicode). If needed
  you might want to convert the datatypes of character variables in this procedure to their respective unicode counterparts
  like nchar and nvarchar
  

Example 1: To generate INSERT statements for table 'titles':
  
  EXEC sp_generate_inserts 'titles'

Example 2:  To ommit the column list in the INSERT statement: (Column list is included by default)
  IMPORTANT: If you have too many columns, you are advised to ommit column list, as shown below,
  to avoid erroneous results
  
  EXEC sp_generate_inserts 'titles', @include_column_list = 0

Example 3: To generate INSERT statements for 'titlesCopy' table from 'titles' table:

  EXEC sp_generate_inserts 'titles', 'titlesCopy'

Example 4: To generate INSERT statements for 'titles' table for only those titles 
  which contain the word 'Computer' in them:
  NOTE: Do not complicate the FROM or WHERE clause here. It's assumed that you are good with T-SQL if you are using this parameter

  EXEC sp_generate_inserts 'titles', @from = "from titles where title like '%Computer%'"

Example 5:  To specify that you want to include TIMESTAMP column's data as well in the INSERT statement:
  (By default TIMESTAMP column's data is not scripted)

  EXEC sp_generate_inserts 'titles', @include_timestamp = 1

Example 6: To print the debug information:
  
  EXEC sp_generate_inserts 'titles', @debug_mode = 1

Example 7:  If you are not the owner of the table, use @owner parameter to specify the owner name
  To use this option, you must have SELECT permissions on that table

  EXEC sp_generate_inserts Nickstable, @owner = 'Nick'

Example 8:  To generate INSERT statements for the rest of the columns excluding images
  When using this otion, DO NOT set @include_column_list parameter to 0.

  EXEC sp_generate_inserts imgtable, @ommit_images = 1

Example 9:  To generate INSERT statements excluding (ommiting) IDENTITY columns:
  (By default IDENTITY columns are included in the INSERT statement)

  EXEC sp_generate_inserts mytable, @ommit_identity = 1

Example 10:  To generate INSERT statements for the TOP 10 rows in the table:
  
  EXEC sp_generate_inserts mytable, @top = 10

Example 11:  To generate INSERT statements with only those columns you want:
  
  EXEC sp_generate_inserts titles, @cols_to_include = "'title','title_id','au_id'"

Example 12:  To generate INSERT statements by omitting certain columns:
  
  EXEC sp_generate_inserts titles, @cols_to_exclude = "'title','title_id','au_id'"

Example 13: To avoid checking the foreign key constraints while loading data with INSERT statements:
  
  EXEC sp_generate_inserts titles, @disable_constraints = 1

Example 14:  To exclude computed columns from the INSERT statement:
  EXEC sp_generate_inserts MyTable, @ommit_computed_cols = 1
***********************************************************************************************************/

SET NOCOUNT ON

--Making sure user only uses either @cols_to_include or @cols_to_exclude
IF ((@cols_to_include IS NOT NULL) AND (@cols_to_exclude IS NOT NULL))
 BEGIN
  RAISERROR('Use either @cols_to_include or @cols_to_exclude. Do not use both the parameters at once',16,1)
  RETURN -1 --Failure. Reason: Both @cols_to_include and @cols_to_exclude parameters are specified
 END

--Making sure the @cols_to_include and @cols_to_exclude parameters are receiving values in proper format
IF ((@cols_to_include IS NOT NULL) AND (PATINDEX('''%''',@cols_to_include) = 0))
 BEGIN
  RAISERROR('Invalid use of @cols_to_include property',16,1)
  PRINT 'Specify column names surrounded by single quotes and separated by commas'
  PRINT 'Eg: EXEC sp_generate_inserts titles, @cols_to_include = "''title_id'',''title''"'
  RETURN -1 --Failure. Reason: Invalid use of @cols_to_include property
 END

IF ((@cols_to_exclude IS NOT NULL) AND (PATINDEX('''%''',@cols_to_exclude) = 0))
 BEGIN
  RAISERROR('Invalid use of @cols_to_exclude property',16,1)
  PRINT 'Specify column names surrounded by single quotes and separated by commas'
  PRINT 'Eg: EXEC sp_generate_inserts titles, @cols_to_exclude = "''title_id'',''title''"'
  RETURN -1 --Failure. Reason: Invalid use of @cols_to_exclude property
 END


--Checking to see if the database name is specified along wih the table name
--Your database context should be local to the table for which you want to generate INSERT statements
--specifying the database name is not allowed
IF (PARSENAME(@table_name,3)) IS NOT NULL
 BEGIN
  RAISERROR('Do not specify the database name. Be in the required database and just specify the table name.',16,1)
  RETURN -1 --Failure. Reason: Database name is specified along with the table name, which is not allowed
 END

--Checking for the existence of 'user table' or 'view'
--This procedure is not written to work on system tables
--To script the data in system tables, just create a view on the system tables and script the view instead

IF @owner IS NULL
 BEGIN
  IF ((OBJECT_ID(@table_name,'U') IS NULL) AND (OBJECT_ID(@table_name,'V') IS NULL)) 
   BEGIN
    RAISERROR('User table or view not found.',16,1)
    PRINT 'You may see this error, if you are not the owner of this table or view. In that case use @owner parameter to specify the owner name.'
    PRINT 'Make sure you have SELECT permission on that table or view.'
    RETURN -1 --Failure. Reason: There is no user table or view with this name
   END
 END
ELSE
 BEGIN
  IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = @table_name AND (TABLE_TYPE = 'BASE TABLE' OR TABLE_TYPE = 'VIEW') AND TABLE_SCHEMA = @owner)
   BEGIN
    RAISERROR('User table or view not found.',16,1)
    PRINT 'You may see this error, if you are not the owner of this table. In that case use @owner parameter to specify the owner name.'
    PRINT 'Make sure you have SELECT permission on that table or view.'
    RETURN -1 --Failure. Reason: There is no user table or view with this name  
   END
 END

--Variable declarations
DECLARE  @Column_ID int,   
  @Column_List varchar(8000), 
  @Column_Name varchar(128), 
  @Start_Insert varchar(786), 
  @Data_Type varchar(128), 
  @Actual_Values varchar(8000), --This is the string that will be finally executed to generate INSERT statements
  @IDN varchar(128)  --Will contain the IDENTITY column's name in the table

--Variable Initialization
SET @IDN = ''
SET @Column_ID = 0
SET @Column_Name = ''
SET @Column_List = ''
SET @Actual_Values = ''

IF @owner IS NULL 
 BEGIN
  SET @Start_Insert = 'INSERT INTO ' + '[' + RTRIM(COALESCE(@target_table,@table_name)) + ']' 
 END
ELSE
 BEGIN
  SET @Start_Insert = 'INSERT ' + '[' + LTRIM(RTRIM(@owner)) + '].' + '[' + RTRIM(COALESCE(@target_table,@table_name)) + ']'   
 END


--To get the first column's ID

SELECT @Column_ID = MIN(ORDINAL_POSITION)  
FROM INFORMATION_SCHEMA.COLUMNS (NOLOCK) 
WHERE  TABLE_NAME = @table_name AND
(@owner IS NULL OR TABLE_SCHEMA = @owner)



--Loop through all the columns of the table, to get the column names and their data types
WHILE @Column_ID IS NOT NULL
 BEGIN
  SELECT  @Column_Name = QUOTENAME(COLUMN_NAME), 
  @Data_Type = DATA_TYPE 
  FROM  INFORMATION_SCHEMA.COLUMNS (NOLOCK) 
  WHERE  ORDINAL_POSITION = @Column_ID AND 
  TABLE_NAME = @table_name AND
  (@owner IS NULL OR TABLE_SCHEMA = @owner)



  IF @cols_to_include IS NOT NULL --Selecting only user specified columns
  BEGIN
   IF CHARINDEX( '''' + SUBSTRING(@Column_Name,2,LEN(@Column_Name)-2) + '''',@cols_to_include) = 0 
   BEGIN
    GOTO SKIP_LOOP
   END
  END

  IF @cols_to_exclude IS NOT NULL --Selecting only user specified columns
  BEGIN
   IF CHARINDEX( '''' + SUBSTRING(@Column_Name,2,LEN(@Column_Name)-2) + '''',@cols_to_exclude) <> 0 
   BEGIN
    GOTO SKIP_LOOP
   END
  END

  --Making sure to output SET IDENTITY_INSERT ON/OFF in case the table has an IDENTITY column
  IF (SELECT COLUMNPROPERTY( OBJECT_ID(QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + @table_name),SUBSTRING(@Column_Name,2,LEN(@Column_Name) - 2),'IsIdentity')) = 1 
  BEGIN
   IF @ommit_identity = 0 --Determing whether to include or exclude the IDENTITY column
    SET @IDN = @Column_Name
   ELSE
    GOTO SKIP_LOOP   
  END
  
  --Making sure whether to output computed columns or not
  IF @ommit_computed_cols = 1
  BEGIN
   IF (SELECT COLUMNPROPERTY( OBJECT_ID(QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + @table_name),SUBSTRING(@Column_Name,2,LEN(@Column_Name) - 2),'IsComputed')) = 1 
   BEGIN
    GOTO SKIP_LOOP     
   END
  END
  
  --Tables with columns of IMAGE data type are not supported for obvious reasons
  IF(@Data_Type in ('image'))
   BEGIN
    IF (@ommit_images = 0)
     BEGIN
      RAISERROR('Tables with image columns are not supported.',16,1)
      PRINT 'Use @ommit_images = 1 parameter to generate INSERTs for the rest of the columns.'
      PRINT 'DO NOT ommit Column List in the INSERT statements. If you ommit column list using @include_column_list=0, the generated INSERTs will fail.'
      RETURN -1 --Failure. Reason: There is a column with image data type
     END
    ELSE
     BEGIN
     GOTO SKIP_LOOP
     END
   END

  --Determining the data type of the column and depending on the data type, the VALUES part of
  --the INSERT statement is generated. Care is taken to handle columns with NULL values. Also
  --making sure, not to lose any data from flot, real, money, smallmomey, datetime columns
  SET @Actual_Values = @Actual_Values  +
  CASE 
   WHEN @Data_Type IN ('char','varchar','nchar','nvarchar') 
    THEN 
     'COALESCE('''''''' + REPLACE(RTRIM(' + @Column_Name + '),'''''''','''''''''''')+'''''''',''NULL'')'
   WHEN @Data_Type IN ('datetime','smalldatetime') 
    THEN 
     'COALESCE('''''''' + RTRIM(CONVERT(char,' + @Column_Name + ',109))+'''''''',''NULL'')'
   WHEN @Data_Type IN ('uniqueidentifier') 
    THEN  
     'COALESCE('''''''' + REPLACE(CONVERT(char(255),RTRIM(' + @Column_Name + ')),'''''''','''''''''''')+'''''''',''NULL'')'
   WHEN @Data_Type IN ('text','ntext') 
    THEN  
     'COALESCE('''''''' + REPLACE(CONVERT(char(8000),' + @Column_Name + '),'''''''','''''''''''')+'''''''',''NULL'')'     
   WHEN @Data_Type IN ('binary','varbinary') 
    THEN  
     'COALESCE(RTRIM(CONVERT(char,' + 'CONVERT(int,' + @Column_Name + '))),''NULL'')'  
   WHEN @Data_Type IN ('timestamp','rowversion') 
    THEN  
     CASE 
      WHEN @include_timestamp = 0 
       THEN 
        '''DEFAULT''' 
       ELSE 
        'COALESCE(RTRIM(CONVERT(char,' + 'CONVERT(int,' + @Column_Name + '))),''NULL'')'  
     END
   WHEN @Data_Type IN ('float','real','money','smallmoney')
    THEN
     'COALESCE(LTRIM(RTRIM(' + 'CONVERT(char, ' +  @Column_Name  + ',2)' + ')),''NULL'')' 
   ELSE 
    'COALESCE(LTRIM(RTRIM(' + 'CONVERT(char, ' +  @Column_Name  + ')' + ')),''NULL'')' 
  END   + '+' +  ''',''' + ' + '
  
  --Generating the column list for the INSERT statement
  SET @Column_List = @Column_List +  @Column_Name + ',' 

  SKIP_LOOP: --The label used in GOTO

  SELECT  @Column_ID = MIN(ORDINAL_POSITION) 
  FROM  INFORMATION_SCHEMA.COLUMNS (NOLOCK) 
  WHERE  TABLE_NAME = @table_name AND 
  ORDINAL_POSITION > @Column_ID AND
  (@owner IS NULL OR TABLE_SCHEMA = @owner)


 --Loop ends here!
 END

--To get rid of the extra characters that got concatenated during the last run through the loop
SET @Column_List = LEFT(@Column_List,len(@Column_List) - 1)
SET @Actual_Values = LEFT(@Actual_Values,len(@Actual_Values) - 6)

IF LTRIM(@Column_List) = '' 
 BEGIN
  RAISERROR('No columns to select. There should at least be one column to generate the output',16,1)
  RETURN -1 --Failure. Reason: Looks like all the columns are ommitted using the @cols_to_exclude parameter
 END

--Forming the final string that will be executed, to output the INSERT statements
IF (@include_column_list <> 0)
 BEGIN
  SET @Actual_Values = 
   'SELECT ' +  
   CASE WHEN @top IS NULL OR @top < 0 THEN '' ELSE ' TOP ' + LTRIM(STR(@top)) + ' ' END + 
   '''' + RTRIM(@Start_Insert) + 
   ' ''+' + '''(' + RTRIM(@Column_List) +  '''+' + ''')''' + 
   ' +''VALUES(''+ ' +  @Actual_Values  + '+'')''' + ' ' + 
   COALESCE(@from,' FROM ' + CASE WHEN @owner IS NULL THEN '' ELSE '[' + LTRIM(RTRIM(@owner)) + '].' END + '[' + rtrim(@table_name) + ']' + '(NOLOCK)')
 END
ELSE IF (@include_column_list = 0)
 BEGIN
  SET @Actual_Values = 
   'SELECT ' + 
   CASE WHEN @top IS NULL OR @top < 0 THEN '' ELSE ' TOP ' + LTRIM(STR(@top)) + ' ' END + 
   '''' + RTRIM(@Start_Insert) + 
   ' '' +''VALUES(''+ ' +  @Actual_Values + '+'')''' + ' ' + 
   COALESCE(@from,' FROM ' + CASE WHEN @owner IS NULL THEN '' ELSE '[' + LTRIM(RTRIM(@owner)) + '].' END + '[' + rtrim(@table_name) + ']' + '(NOLOCK)')
 END 

--Determining whether to ouput any debug information
IF @debug_mode =1
 BEGIN
  PRINT '/*****START OF DEBUG INFORMATION*****'
  PRINT 'Beginning of the INSERT statement:'
  PRINT @Start_Insert
  PRINT ''
  PRINT 'The column list:'
  PRINT @Column_List
  PRINT ''
  PRINT 'The SELECT statement executed to generate the INSERTs'
  PRINT @Actual_Values
  PRINT ''
  PRINT '*****END OF DEBUG INFORMATION*****/'
  PRINT ''
 END
  
PRINT '--INSERTs generated by ''sp_generate_inserts'' stored procedure written by Vyas'
PRINT '--Build number: 22'
PRINT '--Problems/Suggestions? Contact Vyas @ vyaskn@hotmail.com'
PRINT '--http://vyaskn.tripod.com'
PRINT ''
PRINT 'SET NOCOUNT ON'
PRINT ''


--Determining whether to print IDENTITY_INSERT or not
IF (@IDN <> '')
 BEGIN
  PRINT 'SET IDENTITY_INSERT ' + QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + QUOTENAME(@table_name) + ' ON'
  PRINT 'GO'
  PRINT ''
 END


IF @disable_constraints = 1 AND (OBJECT_ID(QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + @table_name, 'U') IS NOT NULL)
 BEGIN
  IF @owner IS NULL
   BEGIN
    SELECT  'ALTER TABLE ' + QUOTENAME(COALESCE(@target_table, @table_name)) + ' NOCHECK CONSTRAINT ALL' AS '--Code to disable constraints temporarily'
   END
  ELSE
   BEGIN
    SELECT  'ALTER TABLE ' + QUOTENAME(@owner) + '.' + QUOTENAME(COALESCE(@target_table, @table_name)) + ' NOCHECK CONSTRAINT ALL' AS '--Code to disable constraints temporarily'
   END

  PRINT 'GO'
 END

PRINT ''
PRINT 'PRINT ''Inserting values into ' + '[' + RTRIM(COALESCE(@target_table,@table_name)) + ']' + ''''


--All the hard work pays off here!!! You'll get your INSERT statements, when the next line executes!
EXEC (@Actual_Values)

PRINT 'PRINT ''Done'''
PRINT ''


IF @disable_constraints = 1 AND (OBJECT_ID(QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + @table_name, 'U') IS NOT NULL)
 BEGIN
  IF @owner IS NULL
   BEGIN
    SELECT  'ALTER TABLE ' + QUOTENAME(COALESCE(@target_table, @table_name)) + ' CHECK CONSTRAINT ALL'  AS '--Code to enable the previously disabled constraints'
   END
  ELSE
   BEGIN
    SELECT  'ALTER TABLE ' + QUOTENAME(@owner) + '.' + QUOTENAME(COALESCE(@target_table, @table_name)) + ' CHECK CONSTRAINT ALL' AS '--Code to enable the previously disabled constraints'
   END

  PRINT 'GO'
 END

PRINT ''
IF (@IDN <> '')
 BEGIN
  PRINT 'SET IDENTITY_INSERT ' + QUOTENAME(COALESCE(@owner,USER_NAME())) + '.' + QUOTENAME(@table_name) + ' OFF'
  PRINT 'GO'
 END

PRINT 'SET NOCOUNT OFF'


SET NOCOUNT OFF
RETURN 0 --Success. We are done!
END

GO

PRINT 'Created the procedure'
GO


--Turn system object marking off
EXEC master.dbo.sp_MS_upd_sysobj_category 2
GO

PRINT 'Granting EXECUTE permission on sp_generate_inserts to all users'
GRANT EXEC ON sp_generate_inserts TO public

SET NOCOUNT OFF
GO

PRINT 'Done'

Read More

SQL Server versions are:

Version Number	SQL Server version
2007.100.1600.0	SQL Server 2008 RTM
2007.100.2531.0	SQL Server 2008 Service Pack 1
2005.90.1399	SQL Server 2005 RTM
2005.90.2047	SQL Server 2005 Service Pack 1
2005.90.3042	SQL Server 2005 Service Pack 2
2005.90.4035	SQL Server 2005 Service Pack 3
2000.80.194.0	SQL Server 2000 RTM
2000.80.384.0	SQL Server 2000 SP1
2000.80.534.0	SQL Server 2000 SP2
2000.80.760.0	SQL Server 2000 SP3
2000.80.760.0	SQL Server 2000 SP3a
2000.8.00.2039	SQL Server 2000 SP4

See http://support.microsoft.com/kb/321185

To find the version of SQL Server try running the following SQL query:

SELECT SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')

Read More

If you want to list all the data and log files plus their sizes try the following SQL:

USE master ; 
GO 

SELECT  db.dbid AS 'DB ID', 
        db.name AS 'Database Name', 
        af.name AS 'Logical Name', 
        af.[size] as 'File Size (in 8-kilobyte (KB) pages)', 
        (((CAST(af.[size] AS DECIMAL(18, 4)) * 8192) / 1024) / 1024) AS 'File Size (MB)', 
        ((((CAST(af.[size] AS DECIMAL(18, 4)) * 8192) / 1024) / 1024) 
          / 1024 ) AS 'File Size (GB)', 
        af.filename AS 'Physical Name' 
FROM    sys.sysdatabases db 
        INNER JOIN sys.sysaltfiles af ON db.dbid = af.dbid 
WHERE   [fileid] IN (1, 2);

Read More

To add time to a date use the DATEADD Transact-SQL statement.

DATEADD (datepart , number, date )

The date parts can be:

datepart	Abbreviations
year	yy, yyyy
quarter	qq, q
month	mm, m
dayofyear	dy, y
day	dd, d
week	wk, ww
weekday	dw, w
hour	hh
minute	mi, n
second	ss, s
millisecond	ms
microsecond	mcs
nanosecond	ns

For example:

SELECT DATEADD(dd, 12, "29 March 1964")

To get the date at midnight you can use the following:

SELECT DATEADD(dd,0, DATEDIFF(dd,0, GETDATE())) as date_at_midnight

See http://msdn.microsoft.com/en-us/library/ms186819.aspx for details.

Read More

If you want to run a query against the results of a stored procedure a simple solution is to use a temporary table and the “INSERT INTO” SQL syntax. For example:

CREATE TABLE #MyTempTable (
     some_field varchar(10),
     some_other_field varchar(40)
)

INTERT INTO #MyTempTable
     EXEC my_stored_procedure

SELECT * FROM #MyTempTable WHERE some_field LIKE '%some value%'
    
DROP TABLE #MyTempTable
GO

Read More

To identify full-text indexes on a server run:

SELECT 
  object_id, 
  OBJECT_NAME(OBJECT_ID) AS ObjectName, 
  is_enabled, 
  has_crawl_completed, 
  crawl_type, 
  crawl_type_desc, 
  * 
FROM sys.fulltext_indexes 

To enable an index run:

ALTER FULLTEXT INDEX ON [ObjectNameHere] ENABLE

To disable an index run:

ALTER FULLTEXT INDEX ON [ObjectNameHere] DISABLE

Note that [ObjectNameHere] should be replaced with the object name identified from the results of the first query.

Read More

If you use the SQL Server Query Profiler and log results to a table you can get some basic stats from the results with these queries:

-- Find most frequently used queries 
SELECT DISTINCT 
    cast(textdata as varchar(150)) as textdata, 
    avg(duration) as avg_duration, 
    count(duration) as Occurences 
FROM 
    [ProfilerTableNameHere] 
GROUP BY 
    Cast(textdata as VarChar(150)) 
ORDER BY 
    count(duration)desc 

-- Find most inefficient queries 
SELECT DISTINCT 
    cast(textdata as varchar(150)) as textdata, 
    avg(duration) as avg_duration, 
    count(duration) as Occurences 
FROM 
    [ProfilerTableNameHere] 
GROUP BY 
    Cast(textdata as VarChar(150)) 
ORDER BY 
    Avg(duration)desc

Read More

If you need to identify the tables in a database including their row counts, column counts and data size try the following SQL:

USE [DatabaseNameHere] 

CREATE TABLE #temp ( 
  table_name sysname , 
  row_count INT, 
  reserved_size VARCHAR(50), 
  data_size VARCHAR(50), 
  index_size VARCHAR(50), 
  unused_size VARCHAR(50)) 

SET NOCOUNT ON 

INSERT #temp 
EXEC sp_msforeachtable 'sp_spaceused "?"' 

SELECT 
  a.table_name, 
  a.row_count, 
  COUNT(*) AS col_count, 
  a.data_size 
FROM 
  #temp a 
INNER JOIN 
  information_schema.columns b 
ON a.table_name collate database_default 
= b.table_name collate database_default 
GROUP BY 
  a.table_name, 
  a.row_count, 
  a.data_size 
ORDER BY 
  CAST(REPLACE(a.data_size, ' KB', '') AS integer) DESC 

DROP TABLE #temp

Read More

To find a Stored Procedure containing specific text run the following SQL:

SELECT OBJECT_NAME(object_id) 
FROM sys.sql_modules 
WHERE
definition LIKE '%[text to search for here]%' AND
OBJECTPROPERTY(object_id, 'IsProcedure') = 1 
ORDER BY 1

Read More

To find the constraints for  a given database column (without using the built-in SQL Managament Studio features) run the following SQL:

SELECT OBJECT_NAME(constid) 
FROM sysconstraints 
WHERE id = OBJECT_ID('table name here')
AND colid = 
    (
        SELECT colid 
        FROM syscolumns 
        WHERE id = OBJECT_ID('table name here') 
        AND name = 'column name here'
    )

Read More

To find the XML indexes in a database:

SELECT * FROM [sys].xml_indexes

To identify the primary XML index:

SELECT [sxi].name "Primary index name", [so].name "Table name" 
FROM [sys].xml_indexes sxi 
JOIN [sys].objects so ON (so.object_id = sxi.object_id) 
WHERE using_xml_index_id IS NULL

To identify the secondary XML indexes:

SELECT sxi.name "Secondary index name", so.name "Table name" 
FROM [sys].xml_indexes sxi 
JOIN [sys].objects so ON (so.object_id = sxi.object_id) 
WHERE using_xml_index_id IS NOT NULL; 

To rebuild an XML index:

ALTER INDEX [XML index name here] ON [table name here] REBUILD

Read More

When restoring databases from backups on a developer machine you will probably need to create Logins for the application you are working on. The Logins will correspond to the Users used by the application but because the original Logins were created on a different server the Logins created on your development machine won't be correctly associated with the Users in the database.

To reassociate a Login with a database User run the following SQL:

EXEC sp_change_users_login 'Update_One', '[user name here]', '[login name here]'

More information and options here: http://msdn.microsoft.com/en-us/library/ms174378.aspx

Read More

MSDN documentation defines SET ARITHABORT as follows:

"Terminates a query when an overflow or divide-by-zero error occurs during query execution.
...
If SET ARITHABORT is ON, these error conditions cause the query or batch to terminate. If the errors occur in a transaction, the transaction is rolled back. If SET ARITHABORT is OFF and one of these errors occurs, a warning message is displayed, and NULL is assigned to the result of the arithmetic operation."

Note that this setting can have significant effect on query performance. It is quite possible for a query to run very quickly in SQL Management Studio but very slowly when run from code (e.g. ADO.Net). A possible explanation is that ADO.Net runs with ARITHABORT set to OFF whereas SQL Management Studio runs with it ON.

If a query is running very slowly from code it is worth checking if ARITHABORT should be set to on.

Read More