Counter of Festivals

Ashok Blog for SQL Learners and Beginners and Experts

Tuesday 11 December 2018

Cannot resolve collation conflict between '' and '' in equal to operation

Cannot resolve collation conflict between '' and '' in equal to operation

Collations and tempdb

The tempdb database is built every time SQL Server is started and has the same default collation as the model database. This is typically the same as the default collation of the instance. If you create a user database and specify a different default collation than model, the user database has a different default collation than tempdb. All temporary stored procedures or temporary tables are created and stored in tempdb. This means that all implicit columns in temporary tables and all coercible-default constants, variables, and parameters in temporary stored procedures have collations that are different from comparable objects created in permanent tables and stored procedures.
This could lead to problems with a mismatch in collations between user-defined databases and system database objects. For example, an instance of SQL Server uses the Latin1_General_CS_AS collation and you execute the following statements:
CREATE DATABASE TestDB COLLATE Estonian_CS_AS;  
USE TestDB;  
CREATE TABLE TestPermTab (PrimaryKey int PRIMARY KEY, Col1 nchar );  
In this system, the tempdb database uses the Latin1_General_CS_AS collation with code page 1252, and TestDB and TestPermTab.Col1 use the Estonian_CS_AS collation with code page 1257. For example:
USE TestDB;  
GO  
-- Create a temporary table with the same column declarations  
-- as TestPermTab  
CREATE TABLE #TestTempTab (PrimaryKey int PRIMARY KEY, Col1 nchar );  
INSERT INTO #TestTempTab  
         SELECT * FROM TestPermTab;  
GO  
With the previous example, the tempdb database uses the Latin1_General_CS_AS collation, and TestDB and TestTab.Col1 use the Estonian_CS_AS collation. For example:
SELECT * FROM TestPermTab AS a INNER JOIN #TestTempTab on a.Col1 = #TestTempTab.Col1;  
Because tempdb uses the default server collation and TestPermTab.Col1 uses a different collation, SQL Server returns this error: "Cannot resolve collation conflict between 'Latin1_General_CI_AS_KS_WS' and 'Estonian_CS_AS' in equal to operation."
To prevent the error, you can use one of the following alternatives:
  • Specify that the temporary table column use the default collation of the user database, not tempdb. This enables the temporary table to work with similarly formatted tables in multiple databases, if that is required of your system.
    CREATE TABLE #TestTempTab  
       (PrimaryKey int PRIMARY KEY,  
        Col1 nchar COLLATE database_default  
       );  
    
  • Specify the correct collation for the #TestTempTab column:
    CREATE TABLE #TestTempTab  
       (PrimaryKey int PRIMARY KEY,  
        Col1 nchar COLLATE Estonian_CS_AS  
       );  

Set or Change the Column Collation


You can override the database collation for charvarchartextncharnvarchar, and ntext data by specifying a different collation for a specific column of a table and using one of the following:
  • The COLLATE clause of CREATE TABLE and ALTER TABLE. For example:
    CREATE TABLE dbo.MyTable  
      (PrimaryKey   int PRIMARY KEY,  
       CharCol      varchar(10) COLLATE French_CI_AS NOT NULL  
      );  
    GO  
    ALTER TABLE dbo.MyTable ALTER COLUMN CharCol  
                varchar(10)COLLATE Latin1_General_CI_AS NOT NULL;  
    GO  
    
  • SQL Server Management Studio. For more information, Collation and Unicode Support.
  • Using the Column.Collation property in SQL Server Management Objects (SMO).
    You cannot change the collation of a column that is currently referenced by any one of the following:
  • A computed column
  • An index
  • Distribution statistics, either generated automatically or by the CREATE STATISTICS statement
  • A CHECK constraint
  • A FOREIGN KEY constraint
    When you work with tempdb, the COLLATE clause includes a database_default option to specify that a column in a temporary table uses the collation default of the current user database for the connection instead of the collation of tempdb.

Collations and text Columns

You can insert or update values in a text column whose collation is different from the code page of the default collation of the database. SQL Server implicitly converts the values to the collation of the column.
Change Collation of a SQL Server Table Column using T-SQL:
First find current column collation:
USE AdventureWorks
GO
Select TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLLATION_NAME, DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'dbo'
AND TABLE_NAME = 'col_test'
AND COLUMN_NAME = 'Text'

-------------- Disable all constraints for database

EXEC sp_msforeachtable "ALTER TABLE ? NOCHECK CONSTRAINT all"

----------------------- Enable all constraints for database

EXEC sp_msforeachtable "ALTER TABLE ? WITH CHECK CHECK CONSTRAINT all"

---------To find column name of collation

Select TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, COLLATION_NAME, DATA_TYPE
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'dbo'
AND TABLE_NAME = 'col_test

----Change Collation of a SQL Server Table Column:
USE databasename
GO
ALTER TABLE [dbo].[col_test]
ALTER COLUMN Text NVARCHAR(10)
COLLATE SQL_Latin1_General_CP1_CS_AS NULL


To change the database collation

  1. Connect to the Database Engine.
  2. From the Standard bar, click New Query.
  3. Copy and paste the following example into the query window and click Execute. This example shows how to use the COLLATEclause in an ALTER DATABASE statement to change the collation name. Execute the SELECT statement to verify the change.
SQL
USE master;  
GO  
ALTER DATABASE MyOptionsTest  
COLLATE French_CI_AS ;  
GO  

--Verify the collation setting.  
SELECT name, collation_name  
FROM sys.databases  
WHERE name = N'MyOptionsTest';  
GO 


https://www.mssqltips.com/sqlservertip/3519/changing-sql-server-collation-after-installation/
http://zarez.net/?p=1866
http://sqlusa.com/bestpractices2005/collatedatabasedefault/
 

Tuesday 4 December 2018

Configure firewall rules before running the TSQL Debugger

Configure firewall rules before running the TSQL Debugger

Windows Firewall rules must be configured to enable Transact-SQL debugging when connected to an instance of the Database Engine that is running on a different computer than the Database Engine Query Editor.

Configuring the Transact-SQL Debugger

The Transact-SQL debugger includes both server-side and client-side components. The server-side debugger components are installed with each instance of the Database Engine from SQL Server 2005 (9.x) Service Pack 2 (SP2) or later. The client-side debugger components are included:
  • When you install the client-side tools from SQL Server 2008 or later.
  • When you install Microsoft Visual Studio 2010 or later.
  • When you install SQL Server Data Tools (SSDT) from the web download.
    There are no configuration requirements to run the Transact-SQL debugger when SQL Server Management Studio or SQL Server Data Tools is running on the same computer as the instance of the SQL Server Database Engine. However, to run the Transact-SQL debugger when connected to a remote instance of the Database Engine, program and port rules in the Windows Firewall must be enabled on both computers. These rules may be created by SQL Server setup. If you get errors attempting to open a remote debugging session, ensure the following firewall rules are defined on your computer.
    Use the Windows Firewall with Advanced Security application to manage the firewall rules. In both Windows 7 and Windows Server 2008 R2, open Control Panel, open Windows Firewall, and select Advanced settings. In Windows Server 2008 R2 you can also open Service Manager, expand Configuration in the left pane, and expand Windows Firewall with Advanced Security.
 Caution
Enabling rules in the Windows Firewall may expose your computer to security threats that the firewall is designed to block. Enabling rules for remote debugging unblocks the ports and programs listed in this topic.

Firewall Rules on the Server

On the computer that is running the instance of the Database Engine, use Windows Firewall with Advanced Security to specify the following information:
  • Add an inbound program rule for sqlservr.exe. You must have a rule for each instance that needs to support remote debugging sessions.
    1. In Windows Firewall with Advanced Security, in the left pane, right-click Inbound Rules, and then select New Rule in the action pane.
    2. In the Rule Type dialog, select Program, and then click Next.
    3. In the Program dialog, select This program path: and enter the full path to sqlservr.exe for this instance. By default, sqlservr.exe is installed in C:\Program Files\Microsoft SQL Server\MSSQL13.InstanceName\MSSQL\Binn, where InstanceName is MSSQLSERVER for the default instance, and the instance name for any named instance.
    4. In the Action dialog, select Allow the connection, and click Next.
    5. In the Profile dialog, select any profiles that describe the computer connection environment when you want to open a debugging session with the instance, and click Next.
    6. In the Name dialog, type a name and description for this rule and click Finish.
    7. In the Inbound Rules list, right click the rule you created, and then select Properties in the action pane.
    8. Select the Protocols and Ports tab.
    9. Select TCP in the Protocol type: box, select RPC Dynamic Ports in the Local port: box, click Apply, and then click OK.
  • Add an inbound program rule for svchost.exe to enable DCOM communications from remote debugger sessions.
    1. In Windows Firewall with Advanced Security, in the left pane, right-click Inbound Rules, and then select New Rule in the action pane.
    2. In the Rule Type dialog, select Program, and then click Next.
    3. In the Program dialog, select This program path: and enter the full path to svchost.exe. By default, svchost.exe is installed in %systemroot%\System32\svchost.exe.
    4. In the Action dialog, select Allow the connection, and click Next.
    5. In the Profile dialog, select any profiles that describe the computer connection environment when you want to open a debugging session with the instance, and click Next.
    6. In the Name dialog, type a name and description for this rule and click Finish.
    7. In the Inbound Rules list, right click the rule you created, and then select Properties in the action pane.
    8. Select the Protocols and Ports tab.
    9. Select TCP in the Protocol type: box, select RPC Endpoint Mapper in the Local port: box, click Apply, and then click OK.
  • If the domain policy requires network communications to be done through IPsec, you must also add inbound rules opening UDP port 4500 and UDP port 500.

Firewall Rules on the Client

On the computer that is running the Database Engine Query Editor, the SQL Server setup or SQL Server Data Tools setup may have configured the Windows Firewall to allow remote debugging.
If you get errors attempting to open a remote debugging session, you can manually configure the program and port exceptions by using Windows Firewall with Advanced Security to configure firewall rules:
  • Add a program entry for svchost:
    1. In Windows Firewall with Advanced Security, in the left pane, right-click Inbound Rules, and then select New Rule in the action pane.
    2. In the Rule Type dialog, select Program, and then click Next.
    3. In the Program dialog, select This program path: and enter the full path to svchost.exe. By default, svchost.exe is installed in %systemroot%\System32\svchost.exe.
    4. In the Action dialog, select Allow the connection, and click Next.
    5. In the Profile dialog, select any profiles that describe the computer connection environment when you want to open a debugging session with the instance, and click Next.
    6. In the Name dialog, type a name and description for this rule and click Finish.
    7. In the Inbound Rules list, right click the rule you created, and then select Properties in the action pane.
    8. Select the Protocols and Ports tab.
    9. Select TCP in the Protocol type: box, select RPC Endpoint Mapper in the Local port: box, click Apply, and then click OK.
  • Add a program entry for the application hosting the Database Engine Query Editor. If you need to open remote debugging sessions from both SQL Server Management Studio and SQL Server Data Tools on the same computer, you must add a program rule for both:
    1. In Windows Firewall with Advanced Security, in the left pane, right-click Inbound Rules, and then select New Rule in the action pane.
    2. In the Rule Type dialog, select Program, and then click Next.
    3. In the Program dialog, select This program path: and enter one of these three values.
      • For SQL Server Management Studio, enter the full path to ssms.exe. By default, ssms.exe is installed in C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\Management Studio.
      • For SQL Server Data Tools enter the full path to devenv.exe:
        1. By default, the devenv.exe for Visual Studio 2010 is in C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE.
        2. By default, the devenv.exe for Visual Studio 2012 is in C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE
        3. You can find the path to ssms.exe from the shortcut you use to launch SQL Server Management Studio. You can find the path to devenv.exe from the shortcut you use to launch SQL Server Data Tools. Right click the shortcut and select Properties. The executable and path are listed in the Target box.
    4. In the Action dialog, select Allow the connection, and click Next.
    5. In the Profile dialog, select any profiles that describe the computer connection environment when you want to open a debugging session with the instance, and click Next.
    6. In the Name dialog, type a name and description for this rule and click Finish.
    7. In the Inbound Rules list, right click the rule you created, and then select Properties in the action pane.
    8. Select the Protocols and Ports tab.
    9. Select TCP in the Protocol type: box, select RPC Dynamic Ports in the Local port: box, click Apply, and then click OK.

Requirements for Starting the Debugger

All attempts to start the Transact-SQL debugger must also meet the following requirements:
  • SQL Server Management Studio or SQL Server Data Tools must be running under a Windows account that is a member of the sysadmin fixed server roll.
  • The Database Engine Query Editor window must be connected by using either a Windows Authentication or SQL Server Authentication login that is a member of the sysadmin fixed server role.
  • The Database Engine Query Editor window must be connected to an instance of the Database Engine from SQL Server 2005 (9.x) Service Pack 2 (SP2) or later. You cannot run the debugger when the Query Editor window is connected to an instance that is in single-user mode.
  • The server needs to communicate back to the client via RPC. The account under which SQL Server service is running should have authenticate permissions to the client

Controlling Statement Execution

In the Transact-SQL debugger, you can specify the following options for executing from the current statement in Transact-SQL code:
  • Run to the next breakpoint.
  • Step into the next statement.
    If the next statement invokes a Transact-SQL stored procedure, function, or trigger, the debugger displays a new Query Editor window that contains the code of the module. The window is in debug mode, and execution pauses on the first statement in the module. You can then move through the module code, for example, by setting breakpoints or stepping through the code.
  • Step over the next statement.
    The next statement is executed. However, if the statement invokes a stored procedure, function, or trigger, the module code runs until it finishes, and the results are returned to the calling code. If you are sure there are no errors in a stored procedure, you can step over it. Execution pauses on the statement that follows the call to the stored procedure, function, or trigger.
  • Step out of a stored procedure, function, or trigger.
    Execution pauses on the statement that follows the call to the stored procedure, function, or trigger.
  • Run from the current location to the current location of the pointer, and ignore all breakpoints.
    The following table lists the various ways in which you can control how statements execute in the Transact-SQL debugger.
ActionPerform action:
Run all statements from the current statement to the next breakpointClick Continue on the Debug menu.

Click the Continue button on the Debug toolbar.
Step into the next statement or moduleClick Step Into on the Debug menu.

Click the Step Into button on the Debug toolbar.

Press F11.
Step over the next statement or moduleClick Step Over on the Debug menu.

Click the Step Over button on the Debug toolbar.

Press F10.
Step out of a moduleClick Step Out on the Debug menu.

Click the Step Out button on the Debug toolbar.

Press SHIFT+F11.
Run to the current cursor locationRight-click in the Query Editor window, and then click Run To Cursor.

Press CTRL+F10.

Ref:
https://docs.microsoft.com/en-us/sql/ssms/scripting/configure-firewall-rules-before-running-the-tsql-debugger?view=sql-server-2017

Wednesday 14 November 2018

9 Exciting New Features with SQL Server 2017

9 Exciting New Features with SQL Server 2017

Microsoft has added a ton of new features in SQL Server 2017, including expanded operating system support, Graph database capability, Python support, and more. Read on to learn about nine of these new features


SQL Server Runs on Windows, Linux and Docker Containers


1 SQL Server Runs on Windows, Linux and Docker Containers

With SQL Server 2017, Microsoft has expanded the operating systems that SQL Server now supports. As of RC2 of SQL Server 2017 it can now be installed on:
  • Windows
  • Red Hat Enterprise Linux 7.3
  • SUSE Enterprise Linux Server v12 SP2
  • Ubuntu 16.04LTS.
  • Docker Engine 1.8+
You will now have options as to what operating systems you will be using to run your instances of SQL Server 2017. This is great news for those of you that are already managing one or more of these environments, or are looking to move SQL Server to one of these non-windows environments.  

Graph Database Capabilities

2 Graph Database Capabilities

Graph database capability was included in SQL Server 2017. A Graph database is a database structure that allows you to store nodes and edges, where nodes are entities, like an Employee, or a Department, and edges are the relationship between nodes like an employee works in a department. With graph database, you can now more easily store many to many relationships between nodes. Storing data that have complex relationships can be stored in a traditional relational structure, but with Graph Database capabilities you can now more easily store those complex relationships between nodes. By having graph database functionality in SQL Server 2017 you can now more efficiently store and query complex relationships between nodes and entities edges. 


Python Support


3 Python Support

With the introduction of Python support in SQL Server 2017, a Data Scientist can now more easily perform machine learning right inside of SQL Server. Having Python integrated in the SQL Server database engine is a big deal and will allow those Data Scientists to perform Python processing without having to move their data outside of SQL Server.  The SQL Server team has made it simple to integrate python code into your applications. They did this by allowing you to execute external python scripts via the “sp_execute_external_script” system stored procedure. Note this system stored procedure was introduced in SQL Server 2016 to support R processing, but now has been expanded with SQL Server 2017 to support Python. With Python integrated into the database engine, DBAs might be concerned about Python stealing valuable resources away from other SQL Server processing. Fret no more, Python resources can be throttled using Resource Governor. To throttle a Python process using Resource Governor, the DBA will need to create a resource pool that throttles external script execution resources and then assign Python processing workloads to the newly create python resource pool. Additionally, there are some logs and views that allow DBAs to see exactly what resources are being consumed by Python. 


Resumable Online Index Rebuild


4 Resumable Online Index Rebuild

If you have a limited maintenance window, or really large indexes that take a long time to rebuild then you will be excited about this new feature. With the introduction of SQL Server 2017, you can now pause your online index rebuild operations, and then restart them later to complete the index rebuild operation. With resumable Online Index rebuilds, DBAs will also be able to restart a failed online index rebuild operation. To minimize the amount of Transaction Log space needed for an online rebuild operation, SQL Server 2017 allows the transaction log to be truncated, while an online index rebuild operation is running. Just like with other new features, Microsoft has provided a new system view named sys.index_resumable_operations that allows you to monitor online index rebuild operations.

New CLR Security Model



5 New CLR Security Model

With the introduction of SQL Server 2017, Microsoft has changed the security model for CLRs. They did this because the Code Access Security (CAS) in the .NET Framework is no longer supported as a security boundary, which means an assembly marked as SAFE may be able to run code that is unsafe, or accesses external system resources. To shore up possible SQL Server security holes around CLRs, SQL Server 2017 has implemented a new configuration option named CLR strict Security and a new system stored procedure named sys.sp_add_trusted_assembly.  The new configuration option, named “CLR strict Security,” when enabled causes all SAFE and EXTERNAL_ACCESS assemblies to be treated as if they are UNSAFE. Note this option is enabled by default. In order to execute SAFE and EXTERNAL_ACCESS CLRs with this new option enabled, you will need to sign all SAFE and EXTERNAL_ACCESS assemblies, as well as mark your CLR databases as trustworthy. I know this sounds like a pain, especially if you have lots of CLRs that are marked as SAFE or EXTERNAL_ACCESS. But there is a work-around. The work-around that was implemented with SQL Server 2017 is to use the new stored procedure named sys.sp_add_trusted_assembly. This stored procedure allows you to add a CLR to the list of trusted assemblies. This stored procedure allows you to whiteliste a CLR. By whitelisting a CLR, SQL Server will execute UNSAFE and EXTERNAL_ACCESS CLRs without you having to sign them or set their databases to trustworthy.  For more information about the two new CLR security features listed above use these links:
CLR strict security 
sys.sp_add_trusted_assembly (Transact-SQL)



Identity Cache


6 Identity Cache

In the old versions of SQL Server, by default SQL Server would clear the identity cache if SQL Server wasn’t shutdown cleanly. Because of this you would end up with gaps in your identity values when SQL Server wasn’t shutdown normally. To resolve that issue Microsoft came up with a new database scoped configuration that allows you to turn off identity caching by database. By turning off identity caching for a database your identity values will no longer have gaps when SQL Server should fail or restart unexpectedly. Since this setting is now database specific, you can have one database that turns off identity caching, while other databases can use the old behavior of managing the identity cache values. 

Adaptive Query Processing


7 Adaptive Query Processing

Executing plans for a given query might change over time as the underline date associated with the query changes. These changes can make the cardinality and row/or count estimates incorrect causing a poor plan to be selected. Adaptive Query processing, in SQL Server 2017, can improve execution time of similar queries over time. You can specify that a database use Adaptive Query Processing by issuing an ALTER DATABASE statement. There are three different types of query optimizations that Adaptive Query Process will consider: Memory Grants, Adaptive Joins, and Interleaving Executions. By adjusting the memory grant value over time Adaptive Query Processing can reduce the amount of memory that spills to disk to drastically improve query performance. The Adaptive Query Processing will adjust a query plan to use either hash join or a nested loop join based on row counts and a threshold.  

Simplify Your Code with New T-SQL Functions

8 Simplify Your Code with New T-SQL Functions

Your coding of T-SQL just got easier with SQL Server 2017. With this new version, Microsoft introduces some new string functions, like TRIM, CONCAT_WS, TRANSLATE, and STRING_AGG to name a few. For example, to remove white space from the beginning and the end of a string column or variable you had to using LTRIM and RTRIM to accomplish this in SQL Server 2016 and below. But now in SQL Server 2017 you can removed white space from the beginning and end of a string with a single execution of the TRIM function call. 

Scaling Out Integration Services

9 Scaling Out Integration Services

With SQL Server 2017, Microsoft implemented a new feature for SSIS called “Scale Out”. With “Scale Out” it allows you to distribute the execution of a package across multiple machines. By having your package run in parallel on one or more machines you will improve the overall perform of your SSIS package. A scaled out SSIS environment can be run On-Premise, as well as on Azure VM. In order to run a package in parallel the “Scale Out” feature needs to be configured. When configuring the “Scale Out” feature you install one Scale Out Master service and one or more Scale Out Worker services. The Scale Out Master service manages communication with Scale Out Worker services to coordinate the parallel execution for an SSIS package. The Scale Out Worker services perform the actual package execution tasks. 

Thanks

Ref:
https://www.databasejournal.com/features/mssql/slideshows/9-new-features-with-sql-server-2017.html