Configuring SQL Server database project

The configuration of SQL Database project is important. It this article we will learn how to create a basic configuration for the project, which would work in 80% of cases.

Configuring Filegroups

Using multiple filegroups allows for a lot of benefits:

1. Separating out internal system data from user data
2. Placing larger tables/indexes on their own filegroup and/or dedicated set of disks for performance
3. Placing archive (or even read-only) data onto their own filegroups and dedicated set of disks to reduce maintenance overhead
4. Ability to bring parts of the database online quickly, no need to wait for a full restore

Having your data separated to several filegroups is important. If you interested into this topic, please read DATABASE FILEGROUPS: JUST LIKE SEATBELTS BUT WITH LESS CHAFING or some other materials, which you quickly find using search engines.

In our test project, we will create two filegroups. One filegroup we will use for users' data, another will we use to store indexes. If you forgot how to add new items in solution, please use Creating stored procedures and additional objects in SQL Server Database Project article for more information.

DataFileGroup.sql code

/*
Do not change the database path or name variables.
Any sqlcmd variables will be properly substituted during 
build and deployment.
*/
ALTER DATABASE [$(DatabaseName)]
     ADD FILEGROUP [DATA]

IndexesFileGroup.sql code

/*
Do not change the database path or name variables.
Any sqlcmd variables will be properly substituted during 
build and deployment.
*/
ALTER DATABASE [$(DatabaseName)]
     ADD FILEGROUP [INDEXES]

I've put files in the root of the project. So the structure of the project looks like on the picture below.

Database project has default filegroup with name 'PRIMARY'. We have to change that and set 'DATA' as default filegroup. To change default file group, please use Set default filegroup for Microsoft Database project article.

Configuring Target platform

Target platform specifies the version of SQL Server that you are targeting with this database project. In Visual Studio 2017 there are few possible options for the target platform: 

• SQL Server 2005
• SQL Server 2008
• SQL Server 2012
• SQL Server 2014
• SQL Server 2016
• Microsoft Azure SQL Database
• etc.

Choose what suits you best. I will keep target platform as SQL Server 2016.

Configuring Default schema

Default schema property specifies the default schema in which both SQLCLR and Transact-SQL objects are created. You can override this setting by specifying schema directly on objects.” In our scenario, we are not going to change default schema.

Configuring Build options (optional)

From my perspective, there is one option among build option which can bring some value to the project. This option is 'Treat Transact-SQL warnings as errors'. This option specifies whether a Transact-SQL warning should cause the build and deployment process to be canceled. If this check box is cleared, warnings appear, but the build and deployment process continues. This setting is specific to the project, not the user, and is stored in the .sqlproj file. Personally, I prefer to set this option to true, because strict rules can reduce potential issues in future. More information about build option you can find using link https://msdn.microsoft.com/en-us/library/hh272681(v=vs.103).aspx#SQLCLR and SQLCLR Build.

Database settings

To change database settings you have to navigate to Project Setting → Database Settings.

Configuring database collation

Collations in SQL Server provide sorting rules, case, and accent sensitivity properties for your data. Collations that are used with character data types such as char and varchar dictate the code page and corresponding characters that can be represented for that data type. Whether you are installing a new instance of SQL Server, restoring a database backup, or connecting server to client databases, it is important that you understand the locale requirements, sorting order, and case and accent sensitivity of the data that you will be working with. 

A collation specifies the bit patterns that represent each character in a data set. Collations also determine the rules that sort and compare data. SQL Server supports storing objects that have different collations in a single database. For non-Unicode columns, the collation setting specifies the code page for the data and which characters can be represented. Data that is moved between non-Unicode columns must be converted from the source code page to the destination code page. Read more about collation https://docs.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support.

To configure database collation navigate to Project Settings → Database Settings → Common → Database Collation.

Configuring compatibility level

A lot of enterprise solutions have a long history, so many of them started on old versions of SQL server. It might be required to setup your project in compatibility level. To do that you have to navigate to Project Settings → Database Settings → Miscellaneous → Compatibility level.

Code analysis

Code analysis is turned off by default. I recommend to activate it and check all 'Treat Warning as Error' checkboxes. To do that navigate to Properties → Code Analysis. (see picture below)

Conclusion

In this article, I tried to provide vision about the configuration of SQL Server Database project. I highlighted the most common and important settings, which should be configured for each project. You can find source code examples using following link https://github.com/aliakseimaniuk/blog-examples/tree/master/WideWorldImporters/04-Configuring.

Compiling SQL Server Database project

One of the great features of SQL Server Database project is the compilation of the project. If the project setup up correctly, developers can use advantages of compilation. Let's assume you have changed the name of the column in some table. You know that this column is used in many places in your database code. It is hard to find all locations where you have to change the name of the column. Compilation of project helps a lot in such scenarios. Just compile your project and Visual Studio will show all places where you have mistakes. To compile the project right-click on project file → Build or Rebuild. After build finishes open 'Error List' window and check if there any error in warning in it.

From previous example http://www.maniuk.net/2017/05/creating-stored-procedures-and-additional-objects-in-sql-server-database-project.html we have some warning which we have to fix. As you can see on the picture below, Visual Studio helps us to identify places which could be the cause of the warning. 

Warning say to us: Severity Code Description Project File Line Suppression State
Warning SQL71502: Procedure: [Application].[CreateRoleIfNonexistent] has an unresolved reference to object [sys].[database_principals].

It means that we have to add a reference to the master database. To add a reference to the master, database you have to navigate to WideWorldImporters project → right-click on References → Add Database Reference → Select System database → Ensure master is selected.

After that, if you recompile your project, all warning will go away. Once again it is critical to use compilation feature of SQL Server Database project. It saves a lot of time for maintenance and issue investigation in future. Source code for this tutorial you can find using link https://github.com/aliakseimaniuk/blog-examples/tree/master/WideWorldImporters/03-Compiling.

Creating stored procedures and additional objects in SQL Server Database Project

In the previous post Creating a new SQL Server Database Project we learned how to create SQL Database project, now we are going to show how to work with some other SQL objects like stored procedures, indexes, key, etc.

Adding stored procedure to the project

When you create stored procedure, you have to keep in mind that you have to put your stored procedure in the appropriate folder related to the particular application domain. In your example, we will create stored procedure in 'Application' domain. So at first, let's create folder 'Stored Procedures' in 'Application' folder. After that right-click on created folder → Add → Stored Procedure.

CreateRoleIfNonexistent.sql stored procedure code

CREATE PROCEDURE [Application].CreateRoleIfNonexistent
@RoleName sysname
WITH EXECUTE AS OWNER
AS
BEGIN
    SET NOCOUNT ON;
    SET XACT_ABORT ON;

    IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = @RoleName AND type = N'R')
    BEGIN
        BEGIN TRY

            DECLARE @SQL nvarchar(max) = N'CREATE ROLE ' + QUOTENAME(@RoleName) + N';'
            EXECUTE (@SQL);

            PRINT N'Role ' + @RoleName + N' created';

        END TRY
        BEGIN CATCH
            PRINT N'Unable to create role ' + @RoleName;
            THROW;
        END CATCH;
    END;
END;

Now we have learned now to create stored procedure in SQL Server Database project.

How to add other SQL objects

Select appropriate folder → right-click on it → Add → New Item.

There are several groups of items:

• Programmability
• Security
• Service Broker
• Server Objects
• SQL CLR
• SQL CLR C#
• Storage
• Table and Views
• User Scripts

If you do not like to search for each group, there is search textbox in the right top corner, and you can use it to find necessary item. Code example for this tutorial could be found using following link https://github.com/aliakseimaniuk/blog-examples/tree/master/WideWorldImporters/02-Add-Stored-Procedure.

Creating a new SQL Server Database Project

In my previous post Migration-base vs State-based database development approach, I've shown two main approaches for database development. Today we will concentrate on state-based approach and learn how to start working with Microsoft Database Project.

Creating new project

Open Visial Studio → click 'File' → 'New' → 'Project'

If you are running Visual Studio 2017 navigate to 'Other languages' → 'SQL server' and select SQL Server Database Project.

After this operation Visual Studio will create an empty database project in the specified folder.

Creating table

To give good understanding for developers about business domains included into the project, the project should be well-structured. It is better to allocate specific schema for each of business domains and put all associated items (stored procedures, tables, triggers, etc.) with the schema in a separate folder. So let's create first business domain with the name 'Application'. At first we have to create folder with name 'Application' in solution. To do that, right-click on solution file → Add → New folder

After you create domain folder, let's create the file with schema. Right-click on 'Application' folder → Add → Script.

Specify an appropriate name for the schema file and make it part of build process. That will allow us to identify issues with SQL code during project compilation.

Application.Schema.sql code

CREATE SCHEMA [Application]
    AUTHORIZATION [dbo];

GO
EXECUTE sp_addextendedproperty 
    @name = N'Description', 
    @value = N'Tables common across the application. Used for categorization and lookup lists, system parameters and people (users and contacts)', 
    @level0type = N'SCHEMA', 
    @level0name = N'Application';

Now we are ready to create our first table. Let's create 'Tables' folder in 'Application' folder and create People table there.

People.sql code

CREATE TABLE [Application].[People] (
    [PersonID]     INT             NOT NULL,
    [FullName]     NVARCHAR (50)   NOT NULL,
    CONSTRAINT [PK_Application_People] PRIMARY KEY CLUSTERED ([PersonID] ASC)
)

GO
EXECUTE sp_addextendedproperty
    @name = N'Description',
    @value = 'Full name for this person',
    @level0type = N'SCHEMA',
    @level0name = N'Application',
    @level1type = N'TABLE',
    @level1name = N'People',
    @level2type = N'COLUMN',
    @level2name = N'FullName';

GO
EXECUTE sp_addextendedproperty
    @name = N'Description',
    @value = 'Numeric ID used for reference to a person within the database',
    @level0type = N'SCHEMA',
    @level0name = N'Application',
    @level1type = N'TABLE',
    @level1name = N'People',
    @level2type = N'COLUMN',
    @level2name = N'PersonID';

GO
EXECUTE sp_addextendedproperty
    @name = N'Description',
    @value = N'People known to the application (staff, customer contacts, supplier contacts)',
    @level0type = N'SCHEMA',
    @level0name = N'Application',
    @level1type = N'TABLE',
    @level1name = N'People';

Finally, we created our first SQL Server Database project. Congrats! Source code related to this tutorial could be found using following link https://github.com/aliakseimaniuk/blog-examples/tree/master/WideWorldImporters/01-Create-Database-Project.

Migration-base vs State-based database development approach

There are two main approaches for the development of SQL databases:

• Transformational (migrations)
• State-based (states)

Transformational or migration-based approach

The idea of the transformational approach is based on database migration from one state to another. After first production release, we will assume that database has its the first state. This state is the starting point for further migrations.

Why do I think that the starting point for migrations should be the first production release? The answer is simple. While the application has not gone into the production stage, it does not make much sense to try to save entered test data. Keeping data complicates development and testing process on earlier stages of project delivery. In this case, it is better to re-create the database each time and insert reference and test data into it. 

The diagram below describes the migration-based approach:

In this method, developers concentrate on writing migrations (M1, M2 .. Mn) from one database state to another (S1, S2 .. Sn). There is a good example of this approach. It is EntityFramework's migration mechanism. For more information, you can navigate and check MSDN article https://msdn.microsoft.com/en-us/library/jj591621(v=vs.113).aspx.

State-based approach

This approach is based on the following paradigm: developers have to maintain the most current snapshot of the database structure. What does it mean? The most convenient way to explain this is to analyze the following example. Let's assume that we have the table below:

CREATE TABLE People (
    PersonID  INT NOT NULL PRIMARY KEY CLUSTERED,
    LastName  VARCHAR(255) NOT NULL,
    FirstName VARCHAR(255) NOT NULL,
    Address   VARCHAR(255) NULL,
    City      VARCHAR(255) NULL
);

At some moment product owner decided to add one more property to the People table. In migration-based world in this particular scenario developer has to create migration script with ALTER TABLE Statement, but in state-base world we need to follow it's the main paradigm, so we need to update table creation script with a new column. Please see below.

CREATE TABLE People (
    PersonID  INT NOT NULL PRIMARY KEY CLUSTERED,
    LastName  VARCHAR(255) NOT NULL,
    FirstName VARCHAR(255) NOT NULL,
    Address   VARCHAR(255) NULL,
    City      VARCHAR(255) NULL,
    Country   VARCHAR(255) NULL
);

So developer needs to keep database structure up-to-date and the tools will do all the work. Cool, right? One of such tools was introduced by Microsoft when they shipped Visual Studio database project.
Let's see how these tools work. On the diagram below, you can find database development schema used by Visual Studio database project.

Microsoft Database project and tools inside of it will be trying to sync the database with current database script. For more info about Microsoft Database Project use https://msdn.microsoft.com/en-us/library/xee70aty(v=vs.100).aspx link.

What approach to choose?

It is difficult to say which approach is better or worse for a particular project. Next, I will give some advantages and disadvantages of each approach, which in my opinion play a key role in the choice of the mechanism for developing a database on the project. And you can decide what is the best for you.

Advantages of migration-based approach

• Understandable workflow of data migration
• Full control on generated SQL-scripts

Disadvantages of migration-based approach

• Need write the tool for deployment script generation or pay for existing one
• Developer's onboarding process require more time for understanding how database upgrade works
• The complex mechanism for keeping the database in the actual state. Developer has to revert database to valid state to perform migration because order of migrations is important

Advantages of state-based

• Gives developers access to the current state of the database
• It is easier to audit and view history of changes in source control
• Force developers to keep actual state of the database in source control
• Database script compilation, developers can identify issues with their code during compilation
• Can detect if somebody did manual changes in the production database and sync it with actual schema in source control
• Build-in mechanism for database refactoring

Disadvantages of state-based

• Developers do not have a lot of control over deployment script generation
• Poor support for user data migration
• Additional review of generated script required, developers have to check generated script before deployment

Set default filegroup for Microsoft Database project

1) First of all you have to create new filegroup. To do that add following file to your database project:

DataFileGroup.sql

/*
Do not change the database path or name variables.
Any sqlcmd variables will be properly substituted during 
build and deployment.
*/
ALTER DATABASE [$(DatabaseName)]
    ADD FILEGROUP [DATA]

2) Navigate to Database project properties in Visual Studio. -> Click "Database settings..."

3) Navigate to "Operational" tab -> Set default filegroup using combo-box with created filegroup on step 1.