zlika's

DENSE_RANK() function in MS SQL Server

zlika — Thu, 05 Aug 2010 04:46:24 +0000

As we can read in Books Online in SQL Server

Ranking functions return a ranking value for each row in a partition. Depending on the function that is used, some rows might receive the same value as other rows. Ranking functions are nondeterministic.

I have created simple example for usage of the Ranking function DENSE_RANK(). We have a database with the addresses of all the residents of a city. Many of them have changed their addresses several times. They have changed them by using web application, because the authorities have decided that’s the most convenient manner for everybody. Down the years, one user may have more then one record (change of address). We need to select only one actual address or only one precedent address per user. We need to partition the table and select only the data we need.

First I create the tables, keys and I populate them.

USE zlika
GO

-- Create Tables and keys
CREATE TABLE [Users]
(
	[ID] INT IDENTITY (1, 1) PRIMARY KEY
	, [FirstName] VARCHAR(50)
	, [LastName] VARCHAR(50)
)
GO

CREATE TABLE [Address]
(
	[ID] INT IDENTITY (1, 1) PRIMARY KEY
	, [UserID] INT
	, [StreetNo] INT
	, [StreetName] VARCHAR(100)
	, [City] VARCHAR(100)
	, [Phone] VARCHAR(25)
	, [DateOfRegistration] DATETIME NULL
)
GO

ALTER TABLE [Address]
WITH CHECK ADD CONSTRAINT
FK_Users_ID FOREIGN KEY
(
	UserID
)
REFERENCES Users
(
	ID
)
GO

ALTER TABLE [Address]
ADD CONSTRAINT DF_Address_DateOfRegistration
	DEFAULT (GETDATE())
	FOR DateOfRegistration
GO

-- Populate Tables
INSERT INTO [Users] (FirstName, LastName) VALUES
('First', 'Smith')
, ('Second', 'George')
, ('Third', 'Thompson')
GO

SELECT * FROM Users
GO

INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (1, 111, 'StreetName1', 'City1', 11111)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (1, 222, 'StreetName2', 'City2', 22222)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (1, 333, 'StreetName3', 'City3', 33333)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (1, 444, 'StreetName4', 'City4', 44444)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (1, 555, 'StreetName5', 'City5', 55555)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (2, 1111, 'StreetName11', 'City11', 1111111111)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (2, 2222, 'StreetName22', 'City22', 2222222222)
WAITFOR DELAY '00:00:01'
INSERT INTO [Address] (UserID, StreetNo, StreetName, City, Phone) VALUES (3, 3, 'StreetName333', 'City333', 333)
GO

SELECT * FROM [Address]
GO

Users table contains three rows as follow:

Users Table

Address table contains eight rows for these three users:

Address Table

We need to partition the table. That means we need to split the data according to some logic. DANSE_RANK() function does it for us. It split users by their ID (UserID), order the partitions descending by date and adds ranks.

SELECT
	A.ID
	, A.UserID
	, U.FirstName
	, U.LastName
	, A.StreetNo
	, A.StreetName
	, A.City
	, A.Phone
	, A.DateOfRegistration
	, DENSE_RANK() OVER (PARTITION BY UserID ORDER BY DateOfRegistration DESC) AS [Rank]

FROM
	[Address] AS A
	INNER JOIN dbo.[Users] AS U
		ON A.UserID = U.ID

--WHERE
--	[Rank] = 2
GO

The result of the query is the data plus a temporary column for ranking.

Adding DENSE_RANK column (click on image)

If I try to reach the result in single query (uncommenting lines 18 and 19), I’ll get “Invalid column name ‘Rank’” error. The reason is that I generate the [Rank] column during the selection of the data and I can’t filter it at this time.

Invalid column name 'Rank'

To get the data that I need, I simulate a View – create a temporary table and read a filtered data from it.

USE zlika
GO

CREATE TABLE #Temp
(
	ID INT
	, [UserID] INT
	, [FirstName] VARCHAR(50)
	, [LastName] VARCHAR(50)
	, [StreetNo] INT
	, [StreetName] VARCHAR(100)
	, [City] VARCHAR(100)
	, [Phone] VARCHAR(25)
	, [DateOfRegistration] DATETIME NULL
	, [Rank] INT
)
GO

INSERT INTO #Temp
SELECT
	A.ID
	, A.UserID
	, U.FirstName
	, U.LastName
	, A.StreetNo
	, A.StreetName
	, A.City
	, A.Phone
	, DateOfRegistration, DENSE_RANK() OVER (PARTITION BY UserID ORDER BY DateOfRegistration DESC) AS [Rank]

FROM
	[Address] AS A
	INNER JOIN dbo.[Users] AS U
		ON A.UserID = U.ID
GO

SELECT
	ID
	, UserID
	, FirstName
	, LastName
	, StreetNo
	, StreetName
	, City
	, Phone
	, DateOfRegistration

FROM
	#Temp

WHERE
	[Rank] = 1

ORDER BY
	UserID
GO

DROP TABLE #Temp
GO

The result is the last address of the residents. If we need the precedent addresses, we’ll filter the [Rank] column to equals 2.

DENSE_RANK result

Félicitations

It’s good to stay only with yourself

zlika — Fri, 30 Jul 2010 02:08:37 +0000

…but not all the time :-)

Félicitations

Yes It Is

zlika — Mon, 12 Jul 2010 15:53:06 +0000

Simple SSIS Project – export to flat file, FTP upload, archive (create the package)

zlika — Sun, 06 Jun 2010 18:26:29 +0000

Microsoft® SQL Server™ Integration Services is the tool that connects the database to the world and works instead of us. I’ll show you how to extract simple data from the database, store the files in a the file system, rename them, upload them on FTP and report the errors to administrator if any. In this first part I will create the package. Next I’ll implement it into SQL Server.

The steps we need are:
1. Create the directories for storing the files – C:\Temp\Source\, C:\Temp\ForUpload\, C:\Temp\Archive\. If they already exist, we use them instead of create them
2. Move the files from Source directory to ForUpload directory with changing of the name (adding date and time)
3. Upload the renamed files on FTP server
4. Move the files to Archive folder

First, create a new SSIS project (File –> New –> Project) in SQL Server Business Intelligence Development Studio.

New SSIS Project

Rename the package.

Rename package

Create Data Source – that’s the place where we will read the data from – MS SQL Server. In Solution Explorer, right click on Data Sources –> New Data Source…

Create Data Source

To use the Data Source, I create a New Connection from the existing Data Source. Right click on Connection Managers –> New Connection From Data Source…

Create New Connection from Data Source

Create the global variables for the directories.

Create global variables

Create the directories – Source, ForUpload, Archive. Drag the File System Task from Toolbox pane.

Create Folders

Right click on File System Task to set the properties of the folders.

Create Folder

For Exporting the data to flat file, first, I create Data Flow Task.

Create Data Flow Task

Data Flow Tab –> Toolbox –> Data Flow Sources (ADO NET Source), Data Flow Transformation Source (Sort), Data Flow Destinations.

Edit ADO NET Source. I have written the query in SSMS, but I can build it on this step.

Edit ADO NET Source

Edit Sort

Edit Sort Transformation

Create Flat File Connection – Delimited

Create Flat File Connection

I chose the delimiter ($) in the Columns section

The next step is to rename the files by adding to filename the date and time. I use Foreach Loop Container to cover all the files in the Source folder. I move them to ForUpload folder and rename them in one step. I do it like this, because I need to know on which step the execution is failed and in that case to handle the rest of operations manually.

Add Foreach Loop Container

Create the local variables for Foreach Loop Container. We need to combine the file path with file name while looping the folder. In the Filename variable we will store the names of the files. To the variable Dir_ForUpload_Filename we will add the date and time.

Create local variables for File Loop Container

I create expressions for Dir_ForUpload_FileName and Dir_Source_FileName variables. When I select the variable, the Properties windows (in the down-right corner of the screen) changes.

Create expression for variable

In the Expression field I write the expresisons:

1. Dir_ForUpload_FileName:

@[User::Dir_ForUpload]
+ SUBSTRING(@[User::FileName], 1, FINDSTRING(@[User::FileName], ".", 1) - 1)
+ "_"
+ SUBSTRING((DT_WSTR, 30)GETDATE(), 1, 4)
+ "-"
+ SUBSTRING((DT_WSTR, 30)GETDATE(), 6, 2)
+ "-"
+ SUBSTRING((DT_WSTR ,30)GETDATE(), 9, 2)
+ "_"
+ SUBSTRING((DT_WSTR, 30)GETDATE(), 12, 2)
+ SUBSTRING((DT_WSTR, 30)GETDATE(), 15, 2)
+ SUBSTRING((DT_WSTR, 30)GETDATE(), 18, 2)
+ SUBSTRING(@[User::FileName], FINDSTRING(@[User::FileName], ".", 1), LEN(@[User::FileName]))

2. Dir_Source_FileName:

@[User::Dir_Source] +  @[User::FileName]

Edit Foreach Loop Container. We mark a folder for loop (C:\Temp\Source\). We loop only the .txt files. In Variable Mappings section we add the FileName variable.

Setting Foreach Loop Container

Setting File System Task (inside the Foreach Loop Container)

Setting File System Task

FTP upload – drag the Foreach Loop Container and FTP task in it. I create local variable FileName – the same as in previous step. In Foreach Loop Editor I choose Foreach File Enumerator, C:\Temp\ForUpload\ in Folder field and *.txt in Files field.

Edit FTP task – I adjust the local and remote paths

Setting FTP task

The next step is loop for placing the files in Archive folder. I won’t explain it. It’s the same as the loop, I used to move and rename from Source to ForUpload folder.

The full schema of the objects looks like this:

All the objects

Next I will implement the package into SSMS, I will create Scheduled Job, which will include operator notification. While executing the Job in real-time working enviroment, I will decide whether to add the Send Mail Task in SSIS package.

Source files

Félicitations

Outlook 2010 Data File Location

zlika — Mon, 31 May 2010 01:14:24 +0000

If you use Outlook 2010 and you prefer to have your data files on external hard drive, you have to point your file location as you add your account manually :-)

Outlook 2010 Data File Location

Félicitations

Congratulations! You’ve passed the Exam! :-)

zlika — Thu, 27 May 2010 03:45:06 +0000

I’ve passed the Exam 70-431: Microsoft® SQL Server™ 2005 – Implementation and Maintenance (Transcript ID: 907652, Access Code: 38959BBD) on May 25, 2010 at Prometric Testing Center in Montreal. What’s next? Microsoft® SQL Server™ + .NET :-)

Félicitations

The difference between WHERE and HAVING filters in SQL Server

zlika — Tue, 27 Apr 2010 02:44:51 +0000

The difference between WHERE and HAVING filters in Transact-SQL queries is that WHERE clause is a filter of all the data, while HAVING filters the GROUP BY data. I’ve prepared a simple example.

USE zlika
GO

-- Create table
CREATE TABLE test (
	Date DATETIME
	, QTY INT
)
GO

-- Populate table
INSERT INTO test (Date, QTY)
SELECT '2009-05-10', 1
UNION ALL SELECT '2009-05-10', 2
UNION ALL SELECT '2009-05-10', 3
UNION ALL SELECT '2009-05-11', 4
UNION ALL SELECT '2009-05-11', 5
UNION ALL SELECT '2009-05-11', 6
UNION ALL SELECT '2009-05-12', 7
UNION ALL SELECT '2009-05-12', 8
UNION ALL SELECT '2009-05-12', 9
GO

-- Check the table
SELECT
	CONVERT(CHAR(10), Date, 103) AS Date
	, QTY

FROM
	test
GO

The content of the table is as follows:

Content of the table

If we GROUP BY column QTY, the result is different sums of the QTY column:
May 10, 2009, 1 + 2 + 3 = 6
May 11, 2009, 4 + 5 + 6 = 15
May 12, 2009, 7 + 8 + 9 = 24

USE zlika
GO

-- GROUP BY (no filters)
SELECT
	CONVERT(CHAR(10), Date, 103) AS Date
	, SUM(QTY) AS QTY

FROM
	test

GROUP BY
	Date
GO

GROPU BY filter

If we exclude the ’6′ in the WHERE clause, it’s gonna be filtered before the grouping of the date. The line No. 6 will be eliminated and the May 11, 2009 will be grouped like 4 + 5 = 9.

USE zlika
GO

-- GROUP BY (WHERE filter)
SELECT
	CONVERT(CHAR(10), Date, 103) AS Date
	, SUM(QTY) AS QTY

FROM
	test

WHERE
	QTY <> 6

GROUP BY
	Date
GO

WHERE filter

If we exclude the ’6′ in the HAVING clause, it’s gonna be filtered after the grouping of the date. The date that has sum of QTY column = 6 will be eliminated and the date May 10, 2009 will miss.

USE zlika
GO

-- GROUP BY (HAVING filter)
SELECT
	CONVERT(CHAR(10), Date, 103) AS Date
	, SUM(QTY) AS QTY

FROM
	test

GROUP BY
	Date

HAVING
	SUM(QTY) <> 6
GO

HAVING filter

Clean up the test table.

USE zlika
GO

-- Clean up
DROP TABLE test
GO

Félicitations

Life is what happens to you while you’re busy making other plans

zlika — Sat, 17 Apr 2010 18:25:01 +0000

John Lennon said:

Life is what happens to you while you’re busy making other plans

on the Beautiful Boy song.

I wrote a schema of my plans and what’s really happening. Now, the scheme, then I’m gonna explain it.

Plans

Explanation: Black – plans, Red – reality (favorable), Blue – reality (unfavorable).

If you think smart, you’ll be able to walk near the red path. It’s no good when the blue path persists.

In the next month I’ll take a walk on the red curve and in the same time I’am preparing myself for a hard exam.

Why?

Félicitations

Chinese Democracy – Sorry

zlika — Mon, 05 Apr 2010 06:05:58 +0000

The interesting thing about me is… Several days ago I received a letter from Microsoft. They presented me a voucher for an exam. Now I’m preparing myself for this exam. It’s a hard one, but I learn a lot :-)

Yesterday my sister told me that Slash is preparing something interesting… From song to song I reached “Chinese Democracy” by Guns N Roses. The only one thing that binds “Chinese Democracy” and Guns N Roses is Axl Rose. He has worked a lot and the result is… Great! Axl proved that he is a great musician! I will be nice surprised if Slash shows us something so good as “Chinese Democracy”

Félicitations

A Perfect Circle – The Package

zlika — Sun, 07 Mar 2010 07:18:47 +0000

If you are not capable to hear the fireflies in the night, don’t push ‘play’ :-)

Félicitations