Showing posts with label user defined functions. Show all posts
Showing posts with label user defined functions. Show all posts

Saturday, September 11, 2010

A MySQL Tidbit: Dynamic Export To_XML Stored Procedure

All XML, All the Time; More Fun MySQL Tidbits – Dynamically Generate XML via Stored Procedure in MySQL

Extensible Markup Language (XML) and database systems, a marriage we are seeing more and more of. So the topics of parsing and manipulating XML, importing and exporting XML files, etcetera using SQL are pretty commonplace here at Experts Exchange.

Consequently, in two of my previous MySQL tidbits, I covered some real questions from EE's Q&A forum:



While writing those articles and participating in the mentioned questions, I was urged by a fellow database expert to write some tips on the importing and exporting of XML, specifically in SQL using MySQL database server. As a result, this particular installment of my SQL tidbits will cover: (1) dynamically constructing XML elements from a given table structure; and (2) exporting a result set to XML file.

The end product will be a technique you can customize and test for your own XML needs. This technique will be shown in a stored procedure. For beginning MySQL readers, I will be providing some background tips -- explanations on some of the inner workings of the SQL code in the procedure...

Read more of "A MySQL Tidbit: Dynamic Export To_XML Stored Procedure" on Experts-Exchange.com/articles...

Enjoy!

Sunday, October 12, 2008

String to Value Algorithm

Well continuing on the thought of conversion routines, here is an example of taking a string value in Transact-SQL on Microsoft SQL Server (may work on other platforms that support same functions) and creating a unique integer value.

Code listing:

DECLARE @text nvarchar(100), @value bigint
SET @text = 'Smith'
SET @value = 0
WHILE LEN(@text) > 0
BEGIN
SET @value = @value + ASCII(LEFT(@text, 1)) * square(LEN(@text))
SET @text = RIGHT(@text, LEN(@text) - 1)
END
SELECT @value

Very simple hopefully. The basis is using the length of the string, create a large integer value that is multiplied against the ascii value of each letter in the string. I was asked this as a question, so I have not explored all of the uses of a function like this; however, as a quick hash it seems to work fine.

Again, just posting for the learning of some of the tools available in SQL like the ASCII and SQUARE functions. Hopefully making the life of some other DBA or programmer a little easier.


References:


Friday, September 19, 2008

Using SQL To Find Work Days In Date Range II

In Using SQL To Find Work Days In Date Range, we created our fn_GetWorkDaysInRange user defined function in Microsoft SQL Server 2005. However, through our research into VB.NET and other simpler algorithms if you have been reading along, if we can do it better why not learn how.

So not to leave well enough alone, here is the code listing our fn_GetWorkDaysInRange revisited:

ALTER FUNCTION [dbo].[fn_GetWorkDaysInRange]
(
@startDate datetime -- first datetime in range
, @endDate datetime -- last datetime in range (can be in past)
, @includeStartDate bit -- flag to include start date as a work day
, @firstWkndDay int -- first day of weekend (e.g. Day(datetime))
, @lastWkndDay int -- last day of weekend (e.g. Day(datetime))
)
RETURNS int
AS
BEGIN
-- variables used in processing
DECLARE @workDays int, @sign int

-- parse input and calculate direction of date range
SET @firstWkndDay = Coalesce(@firstWkndDay, 0)
SET @lastWkndDay = Coalesce(@lastWkndDay, @firstWkndDay)
SET @startDate = Coalesce(@startDate, getdate())
SET @endDate = Coalesce(@endDate, @startDate)
SET @sign = Sign(DateDiff(dd, @startdate, @enddate))

-- set initial value of work days result based on include start date value
-- we use sign so that end dates older than start return negative work days
-- for work days to come up as positive value no matter what, sign usage can be replace by 1
SET @workDays = CASE @includeStartDate WHEN 0 THEN 0 ELSE
CASE
WHEN DatePart(dw, @startDate) IN (@firstWkndDay, @lastWkndDay)
THEN 0
ELSE Case @sign When 0 Then 1 Else @sign End
END
END

-- while end date is not equal to start date add sign (-1/1) number of days
-- and add to work days total if not a weekend
WHILE DateDiff(dd, @startDate, @endDate) <> 0
BEGIN
SET @startDate = DateAdd(dd, @sign, @startDate)
SET @workDays = @workDays +
CASE
WHEN DatePart(dw, @startDate) IN (@firstWkndDay, @lastWkndDay)
THEN 0
ELSE @sign
END
END

-- return working days result to caller
RETURN @workDays
END
The number of lines look very similar, but if you closely inspect this new version there are many changes/simplifications. The table variable is no longer needed, removing further dependence on new version(s) of SQL. The logic is reduced to while loop with two execution lines: increment/decrement date value; add to work days total if the new date value is a working day. The extra code is for readability of case logic.

To have this reflect how clean it really is, we can abstract out the logic for weekend which gets rid of the parameters and set statements for weekend day along with extensive case logic. We could then extend that separate function named something like fn_IsNotWeekDay to include logic to check date against our holiday table returning a bit flagging weekend/holiday. With CLR based user defined functions like we explored using VB.NET, this logic can be as complex as we are capable of coding in either SQL or .NET.

Until the next learning adventure.

Keep the code alive!


Related Articles/References:

Wednesday, September 17, 2008

Using SQL To Find Work Days In Date Range

I had a question come up today for Microsoft SQL Server 2005 on how to calculate the number of working/business days between two dates with a requirement that the answer must function in countries where weekend can be variable two days or even one. My first thought was that is simple. Famous last words!

Eight hours of research later, this article covers the calculation of working days in the date range which appears to work quite nicely. Here is a code listing:

CREATE FUNCTION [dbo].[fn_GetWorkDaysInRange]
(
@startDate datetime -- first datetime in range
, @endDate datetime -- last datetime in range (can be in past)
, @includeStartDate bit -- flag to include start date as a work day
, @firstWkndDay int -- first day of weekend (e.g. Day(datetime))
, @lastWkndDay int -- last day of weekend (e.g. Day(datetime))
)
RETURNS int
AS
BEGIN
-- variables used in processing
DECLARE @workDays int, @sign int
DECLARE @table table (calendarDate datetime, isWorkDay bit)

-- parse input and calculate direction of date range
SET @firstWkndDay = Coalesce(@firstWkndDay, 0)
SET @lastWkndDay = Coalesce(@lastWkndDay, @firstWkndDay)
SET @startDate = Coalesce(@startDate, getdate())
SET @endDate = Coalesce(@endDate, @startDate)
SET @sign = Sign(DateDiff(dd, @startdate, @enddate))

-- insert our starting date
INSERT INTO @table
VALUES (@startDate, Case @includeStartDate When 0 Then 0 Else NULL End)

-- add dates into table from start to end date
IF @sign > 0 -- use sign of date difference to determine direction
BEGIN
WHILE (SELECT MAX(calendarDate) FROM @table) < @enddate
INSERT INTO @table
SELECT DateAdd(dd, 1, MAX(calendarDate)), NULL FROM @table
END
ELSE
BEGIN
WHILE (SELECT MIN(calendarDate) FROM @table) > @enddate
INSERT INTO @table SELECT DateAdd(dd, -1, MIN(calendarDate)), NULL FROM @table
END

-- update table to tag work days
UPDATE @table
SET isWorkDay = CASE
WHEN DatePart(dw, calendarDate) IN (@firstWkndDay, @lastWkndDay)
THEN 0
ELSE 1
END
WHERE isWorkDay IS NULL

-- select the working days from our table into return variable
SELECT @workDays = COUNT(calendarDate) FROM @table WHERE isWorkDay = 1

RETURN (@workDays * @sign)
END
As stated above, this function will remove weekend days between the starting and ending range and thus return number of working days. Once we have that result, we can use a query to retrieve our holidays (or alternatively modify the above to take in a country code and lookup the weekend days and holidays returning the net business days).

Here is an example of this function's usage

-- create holidays table for testing data
CREATE TABLE [dbo].[Holidays](
[day] [datetime],
[holiday] [nvarchar](50) COLLATE SQL_Latin1_General_CP1_CI_AS NULL,
[country] [nvarchar](3) COLLATE SQL_Latin1_General_CP1_CI_AS
) ON [PRIMARY]

INSERT INTO Holidays
SELECT '12/25/2008', 'Christmas', 'USA'
UNION SELECT '12/26/2008', 'Day After Christmas', 'USA'
-- end creation of table for testing data

DECLARE @workDays int, @holidays int
DECLARE @startDate datetime, @endDate datetime

SET @startDate = '12/15/2008'
SET @endDate = '12/29/2008'

SELECT @workDays = dbo.fn_GetWorkDaysInRange(@startDate, @endDate, 0, 7, 1)

SELECT @holiDays = COUNT([day])
FROM [Holidays]
WHERE [country] = 'USA' AND DatePart(dw, [day]) NOT IN (7, 1)
AND [day] BETWEEN @startDate AND @endDate

PRINT (@workDays)
PRINT (@holidays)
PRINT (@workDays - @holidays)

Results come out 10, 2, and 8 for each of the three print statements, respectively. Exactly what we wanted! It is a joy when it all works.

Hopefully this post will save you as long journey, but leave enough uncharted territory to have a little fun with in customizing to your own environment. I have even played with this myself to replace some logic I was using for determining shop working days, so enjoy. For those of you not on Microsoft SQL Server 2005, please keep in mind that other versions of Microsoft SQL Server that support user defined functions should work. Consequently, for other platforms or versions, the structure of this code can probably be manipulated to work in a stored procedure and/or using temporary table instead of a table variable and likewise for other features used not present in your system. The principles should be the same.

Hope this helps and happy coding!

Monday, September 15, 2008

DateSerial In Microsoft SQL Server 2005

Well, in Group By Time: TimeSerial Makes A Return To Microsoft SQL 2005 we explored bringing TimeSerial functionality to SQL Server, but of course we can't stop there as the DateSerial function is pretty useful too.

There are probably a number of different methods to achieve this, but here is what I came up with.

Code listing:


CREATE FUNCTION dbo.DateSerial(@year int, @month int, @day bigint)
RETURNS datetime
AS
BEGIN
DECLARE @date datetime

-- catch invalid year entries and default appropriately
SET @year = CASE
WHEN @year < 1900 Then 1900
When @year > 9999 Then year(getdate())
Else @year End

-- convert date by adding together like yyyymmdd
SET @date = Cast(Cast(@year * 10000 + 101 As varchar) As datetime)
-- Alternative method of parsing year into base date
-- SET @date = Cast('1/1/' + Cast(@year As varchar) As datetime)

-- Add to date the proper months subtracting 1 since we used 1 as start instead of zero.
SET @date = DateAdd(mm, @month - 1, @date)
-- Add to date the proper days subtracting 1 since we used 1 as start instead of zero.
SET @date = DateAdd(dd, @day - 1, @date)

RETURN @date
END
First line is to avoid errors in incorrect starting year value, but can be adjusted according to your own needs. The months and days are added in through simple DateAdd which allows for positive/negative numbers in addition to not being bound by 12 or 31 respectively making this like our TimeSerial function in that it can be used to simply convert a year, month, and day into date or to do some date math on the fly.

Usage:

SELECT dbo.DateSerial(YEAR(GETDATE()), MONTH(GETDATE()), 1 - 35) AS dateSerialized
This will return the date 35 days prior to the first day of the current month in the current year. Moreover, since this solution takes advantage of user defined function only, implementing in SQL 2000 should not be an issue.

So there you have it, DateSerial in SQL Server.

Happy coding!


References:


Sunday, September 07, 2008

Group By Time: TimeSerial Makes A Return To Microsoft SQL 2005

Last time we looked at grouping by time for our customer calls statistics by the half an hour. We ended with this code based on SQL 2005 common table expression syntax.

WITH TableByTime As (
SELECT Cast(DateName(hh, CallDateTime) As Int) As [Sort],
Right('0' + Case When DateName(hh, CallDateTime) = 0 Then '12'
When DateName(hh, CallDateTime) <= 12 Then DateName(hh, CallDateTime)
Else DateName(hh, DateAdd(hh, -12, CallDateTime)) End, 2) + ':' +
Case When DateName(n, CallDateTime) >= 30 Then '30' Else '00' End +
Case When DateName(hh, CallDateTime) < 12 Then 'AM' Else 'PM' End As [Hour],
Customer
FROM CustomerCalls (NoLock)
)
SELECT Hour, Count(*)
FROM TableByTime
GROUP BY Sort, Hour
ORDER BY Sort, Hour
We can be satisfied with this, but why waste a perfectly good opportunity to keep learning. In all seriousness, breakdowns by date/time are statistics I often have to get, so if you are anything like me it would be good to explore making this more reusable and streamlined. Since what we have above is very clean compared to the starting point, what we have left to do is create a function to emulate the TimeSerial function from Microsoft Access that does what our case statements are doing and more.

In summary, TimeSerial, takes in hour in military notation (i.e. 14 for 2PM), minutes, and seconds and translates to appropriate time in format hh:mm:ss with AM/PM indicator. In addition to straight time display, it could do calculations for you based on varying inputs and use of negatives. If you want more information on how TimeSerial functions, see reference for function in Microsoft Access below.

So diving in, we can write a function like this that adds TimeSerial to Microsoft SQL Server using a user defined function.

CREATE FUNCTION dbo.TimeSerial (@hrs int, @min int, @sec bigint)

RETURNS nvarchar(10)

AS

BEGIN

DECLARE @result nvarchar(10), @total bigint, @AMorPM nvarchar(2)

DECLARE @hours int, @minutes int, @seconds int



-- convert everything to seconds handling null params with isnull or coalesce

SET @total = IsNull(@sec,0) + IsNull(@min,0) * 60 + IsNull(@hrs,0) * 3600

SET @total = 86400 + @total % 86400 -- handle negative time relative to midnight



-- calculate the hour portion

SET @hours = 0

IF (@total >= 3600)

BEGIN

SET @hours = floor(@total/3600) % 24

SET @total = @total % 3600

END



-- set am/pm based on hours in HH format

SET @AMorPM = 'PM'

IF @hours < 12

BEGIN

SET @AMorPM = 'AM'

END



-- adjust hours to non-military time

IF @hours > 12 OR @hours = 0

BEGIN

SET @hours = abs(@hours - 12)

END



-- calculate the minutes and seconds portion

SET @minutes = 0

IF (@total >= 60)

BEGIN

SET @minutes = floor(@total/60)

SET @total = @total % 60

END



-- set seconds to remainder

SET @seconds = @total



SET @result = Cast(@hours As nvarchar(2)) + ':' + RIGHT('0'+Cast(@minutes As nvarchar(2)), 2)

SET @result = @result + ':' + RIGHT('0'+Cast(@seconds As nvarchar(2)),2) + @AMorPM



RETURN @result

END
As you will see in the code above, which hopefully speaks for itself as to what it is doing, we can do a little more than just format our time so we abstract out the need for extensive case when logic and give ourselves a handy utility function for our SQL toolkit.

Putting it in place with our original query, we can immediately simplify our syntax to this.

WITH TableByTime As (

SELECT Cast(DateName(hh, CallDateTime) As Int) As [Hr],

FLOOR(Cast(DateName(n, CallDateTime) As Int)/30) * 30 As [Mi],

Customer

FROM CustomerCalls (NoLock)

)

SELECT dbo.TimeSerial(Hr, Mi, 0) As [Hour],

Count(*)

FROM TableByTime

GROUP BY Hr, Mi

ORDER BY Hr, Mi
Aside from the TimeSerial function addition to the code, you will notice a better algorithm to get to group time in 30 minute buckets using floor which gets the lowest integer count of 30 in the number of minutes currently in our time. Since Microsoft SQL will typically do integer division on two numbers that are int datatypes this is probably unnecessary, but I like to be very deliberate in code for nothing else than I will remember what I was thinking when I look at it a year later. The premise here is that we will only ever get 0 or 1 from the division and then multiplying by 30 will give us the correct bucket.

Well we are close to being golden, but we still have a little fluff just to extract the hour and minute portions of time as we have to get as string then cast, so we could write functions for those as well; however, with Microsoft SQL Server, you can utilize ODBC canonical functions for { fn HOUR() } and { fn MINUTE() }.



dbo.TimeSerial({ fn HOUR(GETDATE()) }, FLOOR({ fn MINUTE(GETDATE()) } / 30) * 30, 0) AS HourBucket

As you see above, in combination with our user defined function for TimeSerial the ODBC canonical functions for HOUR and MINUTE make it very streamlined to get our hour bucket in one statement. For sorting purposes, it will probably still be a good idea to use the { fn HOUR() } in Hr column and { fn MINUTE() } in the Mi column in previous code using the common table expression we composed, then just use TimeSerial with those column values.

In conclusion, combining the flexibility of user defined functions and some of the tools in our SQL toolkit with Microsoft SQL Server, we can keep the user community happy with snappy time based reports. The common language runtime (CLR), which we didn't discuss here in detail, is another great means of adding user defined functionality. Anyway, happy coding.


References:


Monday, September 01, 2008

Converting Delimited String To Separate Values In SQL

Ever need to split a comma delimited string for use in SQL? If so, this post will go through a simple Microsoft SQL Server 2005 example of converting delimited text to a SQL table variable.

CREATE FUNCTION [dbo].[split] (@csv nvarchar(max), @delim varchar(1))
RETURNS @entries TABLE (entry nvarchar(100))
AS
BEGIN
DECLARE @commaindex int
SELECT @commaindex = CHARINDEX(@delim, @csv)

IF @commaindex > 0
BEGIN
INSERT INTO @entries
-- insert left side
SELECT LTrim(RTrim(LEFT(@csv, @commaindex-1)))
-- pass right side recursively
UNION ALL
SELECT entry
FROM dbo.split(RIGHT(@csv, LEN(@csv) - @commaindex), @delim)
END
ELSE
BEGIN
INSERT INTO @entries
SELECT LTrim(RTrim(@csv))
END

RETURN
END
The above can be manipulated according to individual need as each entry may need to be more than a 100 character string or you may have a delimiter greater than one character. The key to this solution is the simplicity achieved through use of recursion. Instead of trying to parse through each position that delimiter could be in string from beginning to end, simply keep taking left portion off and passing right portion for further processing.

To take a string and get distinct values, the UNION ALL statement can be changed to UNION or simply select distinct in usage of function which is pretty straight forward as well.

SELECT DISTINCT entry FROM dbo.split('my;delimited;text;to;be;parsed', ';')
As you can see, the string doesn't have to be comma delimited either.

Of course this solution is very basic, but hopefully demonstrates some of the power of using user defined functions and recursion as a part of your SQL toolkit. Well keep learning and keep smiling.