gregs » database

Find duplicates

gregs — Tue, 21 Sep 2010 09:47:50 +0000

Because I have to keep running this script and thus look it up, keeping it here for handy and quick reference

CODE:

select count(id)
from database.dbo.tablename
group by id
having count(id)> 1

Upgrading to Lenny

gregs — Tue, 09 Mar 2010 06:52:56 +0000

It's been while since I attended to my VPS, I decided to spend some time last night upgrading my distribution from Etch to Lenny. This is normally a moment where your heart sinks as quite a few things do tend to go belly up, but I am happy to report that I only came across some minor issues and these were resolved in minutes as opposed to hours.

MySQL : failed to start, complaining about:
PLAIN TEXT
CODE:
1. /etc/init.d/mysql: ERROR: Using expire_logs_days without log_bin crashes
2. the server. See README.Debian.gz
Commenting out the expire_logs_days in the my.cnf file allowed me to restart MySQL.
After the upgrade of Apache, my virtual hosts weren't working. A quick search via Google pointed me to this post - a quick edit of all of my host files and it was all working again.
the php-mysql connector somehow hadn't been upgraded/installed so a quick
PLAIN TEXT
CODE:
1. apt-get install php5-mysql
fixed that problem.
OpenBD : the only thing that remains broken was my tomcat5.5 Open BD install. Tomcat was working fine but Open BD refused to start up complaining about :
PLAIN TEXT
CODE:
1. javax.servlet.ServletException: Open BlueDragon Engine Failed to initialise tags: java.awt.Color
Since I am not really using it, it's not that important, but at some stage I'd like to get it working again. If you have any suggestions, please leave a comment.

SQL to XML

gregs — Wed, 29 Aug 2007 05:12:45 +0000

Just came across this neat trick that uses SQL to generate XML on the fly:

PLAIN TEXT

CODE:

DECLARE @x xml
SET @x=(select geoAreaCode
, geoLevelCode
, geoLevelId
, geoAreaId
from tbl_geoLevel
where uuid =
for xml raw('geoArea'), root('geoAreas'), type)
SELECT @x AS geoAreaInfo

The above will create an XML structure that has a root value of geoAreas and child elements of geoArea for each row found.

Beyond SQL injection attacks

gregs — Wed, 30 Nov -0001 00:00:00 +0000

Courtesy of Slashdot a reminder that you should also worry about other types of injection attacks. Don't you just wish that everybody would play nice?

UPDATE

In a similar vein, Nadav Samet covers off in more detail Cross Site Scripting attacks, how they work and what you can do to protect your site.

LogParser

gregs — Wed, 30 Nov -0001 00:00:00 +0000

Charlie Arehart has a handy little article published in this months CFDJ on using Microsoft's Log Parser tool, which allows you to write SQL like statements against your CFMX log files. Couldn't find a publicly accessible link, but if you keep an eye on his articles page, I am sure he'll update it as soon as it is available.

Update

Charlie dropped me a note last week to let me know that the article can be found on-line here

AUTO INCREMENT

gregs — Wed, 30 Nov -0001 00:00:00 +0000

Since the MySQL Query Browser on my Mac refused to let me set a field as auto increment through the GUI, here's how you can using SQL change an existing column to use auto_increment:

PLAIN TEXT

SQL:

ALTER TABLE `aggregatedWeekly` CHANGE `Id` `Id` INT( 6 ) UNSIGNED NOT NULL AUTO_INCREMENT

UPDATE

I have also recently found out that the following works just as well:

PLAIN TEXT


SQL:




ALTER TABLE `aggregatedWeekly` AUTO_INCREMENT=1

CFMX – MSDE connection refused error

gregs — Wed, 30 Nov -0001 00:00:00 +0000

I just started using MSDE for local development again and after creating my first DSN through the ColdFusion administrator I encountered the following problem:

PLAIN TEXT

CODE:

# Connection verification failed for data source: SMS
java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect
The root cause was that: java.sql.SQLException: [Macromedia][SQLServer JDBC Driver]Error establishing socket. Connection refused: connect

First thing I checked (well second after making sure I had provided the right connection details) was to see if anything was listening port on port 1433 by using the handy "netstats -a" command. And there wasn't, so a quick Google search turned up this gem from Robi Sen. Sadly this tool didn't help me as apparently my version didn't need it, but a comment did point me to a another link. But sadly it didn't help me either, but the comment did give me an idea. When I went to check my network set up for MS SQL server, sure enough TCP/IP was not listed... What followed was a moment of banging my head sharply on the table.

Right changes applied, restarted the server, verify connection and...

PLAIN TEXT

CODE:

Connection verification failed for data source: SMS
java.sql.SQLException: [Macromedia][SQLServer JDBC Driver][SQLServer]Login failed for user 'hermes'. Reason: Not associated with a trusted SQL Server connection.
The root cause was that: java.sql.SQLException: [Macromedia][SQLServer JDBC Driver][SQLServer]Login failed for user 'yourUsername'. Reason: Not associated with a trusted SQL Server connection.

Well this one only took a few seconds to fix: Windows Authentication only was enabled! Simply open the properties of your server once more, check the security tab and you should see that the option for Windows Authentication is selected. Just change that to SQL server and Windows and you are ready to go...

I guess this is very much a case of damned if you do and damned if you don't for Microsoft as they seemed to have tightened up the security for MS SQL server (I won't even go into the nuisance of trying to install MSDE and being prompted to set an SA password before starting with no indication how to do so), but surely we should be given the option at install time to set these things? Or maybe it's just the MSDE?

LIKE and Oracle

gregs — Wed, 30 Nov -0001 00:00:00 +0000

I never spotted this before but the LIKE operator is case sensitive! Which was skewing my search results quite severely. The work around is to put everything into upper case, both the values in the column you are performing the search against as well as the value of the varibale you are searching for.

PLAIN TEXT

CODE:

SELECT col1, col2, col3
FROM table
WHERE UPPER(col4) LIKE '%UPPER(yourValue)%'

Updates from a select

gregs — Wed, 30 Nov -0001 00:00:00 +0000

I have been doing a lot more database based work recently and have learned one or two pretty useful techniques. Here is an extremely useful one: Running an update based on a select statement.

In the past whenever I could have wanted to do a bulk update on a table based on certain criteria, I would have run a select statement to retrieve a bunch of columns first and then programmatically looped over the result set and updated the values. But there is a much, much easier and way more efficient way of doing this. Just take a quick look at the statement below.

PLAIN TEXT

CODE:

UPDATE someUsersTable
SET someUsersTable.EmailPrivate = someOtherJoinTable.PublicEmail
FROM someUsersTable INNER JOIN someOtherJoinTable ON someUsersTable.userID = someOtherJoinTable.userID
WHERE (tbl_Users.userEmailPrivate = '')

It was constructed very much the way I would have normally approached the problem. First I wrote the select statement in Query Analyzer, i.e. SELECT * FROM FROM someUsersTable INNER JOIN someOtherJoinTable ON someUsersTable.userID = someOtherJoinTable.userID WHERE (tbl_Users.userEmailPrivate = ''). This gave me the result set I wanted to work with. Next I quite simply removed the SELECT statement replaced it with a corresponding UPDATE statement. So easy when you know how and the performance should be way better than before hand, because rather than making multiple trips back and forth to the database server from the web application it's all handled by the database server in one swoop!

Accessing a dB on a remote server

gregs — Wed, 30 Nov -0001 00:00:00 +0000

Some of the information we wanted to publish on my previous company's intranet was residing on databases that weren't on the main web server. This initially caused a little headache as we really needed to access that data on the remote NT server and publish it. So how do you get around that problem??

Well the solution to that problem turns out to be incredibly simple. I maped a drive to our web server to the share where the database was held. Then I started up the Cold Fusion Administrator and created a datasource for that database (Db) and went through the process in the normal manner. You'll notice that once you browse for that Db that you can't see the mapped drive. So go back to the previous screen and enter the path manually, something like this:

[code]
{drive letter}\:{share name}\{directory or file name}
[\code]

That's pretty much it. You can now safely create the datasource. Should the Verification process fail go to the advanced properties section of the datasource properties, you may have to specify a user account with sufficient permissions to access that share, i.e. create a CF service account with sufficient permissions and that should do the trick.