Category Archives: Software Development Tips

Things I find that might help some other developer.

Crash, DEP, IE 8, KB977724, Sharepoint 2007, Software Development Tips, Windows Server 2008

Windows Server 2008 IE 8 Hardware DEP Exception After Installing Microsoft Patch KB977724 for Office 2007

2 Comments

Short version – If you run into IE 8 suddenly crashing with a DEP exception after installing Microsoft’s Office Patch KB977724, you need to visit the following link to get a hot fix for KB977724.

Long version –
I spent half a day trying to figure out why I suddenly had IE 8 crashing in Windows Server 2008 while working on Sharepoint 2007 WSS development. The DEP exception would lock up IE 8 with a small dialog box indicating that IE had crashed. I tried the standard stuff – going through the add-ons disabling everything and nothing reliably stopped the crashes – except for running IE with no add-ons loaded. I thought it was related somehow to SharePoint 2007 or possibly an exploit of some sort. A day later I found this post above which doesn’t specifically mention IE 8 – but does work for my situation. I think that SharePoint’s interaction with Office 2007 is causing IE 8 to crash.

Automated Testing, Selenium, Selenium IDE, Selenium RC, Software Development Tips, Software Testing, Windoes 2003 Server

Running Selenium RC Automated Tests Automatically Without a Windows Service

6 Comments

We have a suite of regression tests that we wanted to run automatically after our MSbuild build is complete. I’ve implemented these tests as C# code exported from Selenium IDE and preprocessed by some .NET code that I wrote. The tests are then compiled using MSBuild and executed.  The problem I ran into that wasn’t clearly documented anywhere was how to run a Selenium RC test as part of a build process – which requires the Java based Selenium RC server running on port 4444. Since we run our test as part of a scheduled task, how do we maintain the Java Selenium server running across logins of the scheduled task?

My first instinct was to run the Selenium server as a Windows service and I found a forum post that provided a .NET service to do this. The problem is that when the service runs in the LocalSystem’s account context, IE was problematic for enabling the proxy that Selenium RC uses during the invocation of tests.  Trying to modify the service to run as a legitimate user also didn’t work – since the service was running in a different Windows Station and IE did not run correctly in this mode either. 

It turned out the simple solution was to just invoke the Selenium Server from the build batch file and then terminate the server when I was finished with the test. There is a built in command in the command line interpreter named "taskkill" that does the job for me. Here is the .bat file code that I used – and note that the first line kills any java apps that are running so be careful.  I needed this line to abort a server in progress if the script or tests failed and left a java session hanging. My build machine doesn’t have any other java things going on so this worked for me. I’m using nunit to invoke my C# tests contained in "jalisngtests.dll". I’m also using the "sleep" command from the Windows 2003 Resource Kit.

taskkill /f /im java.exe
sleep 5
start "Selenium Server" java ^
-jar C:\selenium-remote-control-1.0-beta-1\^
selenium-server-1.0-beta-1\selenium-server.jar
copy .\bin\debug\*.dll
sleep 20
nunit-console jalisngtests.dll
taskkill /f /fi "WINDOWTITLE eq Selenium Server"
Automated Testing, JMeter, Selenium, Software Development Tips, Software Testing

Found This Blog Entry: Agile Ajax: Ajax Testing: Doubling Down with Selenium and JMeter

1 Comment

Agile Ajax: Ajax Testing: Doubling Down with Selenium and JMeter

This could be a solution for our automated testing issues.  Use Selenium IDE to record and test and then play back the test while JMeter is watching your HTTP traffic.

Automated Testing, Selenium, Selenium IDE, Software Development Tips, Software Testing

Our Problem with the Selenium IDE and AJAX

1 Comment

We have an application that uses quite a bit of Ajax and this caused problems in recording tests using the Selenium IDE. Each test step was recorded successfully, but because there are web services calls firing in the background on the client (the Ajax stuff), a playback would fail immediately. The fix in general is to use Javascript to check for a completion flag of some sort, but the IDE did not have a provision for inserting code automatically or to delay each test step.  This limited the ability to rerun a recorded test when mistakes are made during the recording process. I’ve been working (with other engineers) on modifying the Selenium IDE code to allow the automatic insertion of Ajax rendering complete Javascript code. However, the latest version of the IDE in SVN has a slow-down control now and that seems to work like a champ on our app.  It allows you to slow the playback down to something that approximates human speed and our app now seems to be happy as we record and then play back.  Thanks to the Selenium IDE developer Shinya Kasatani for this great addition.

I still plan to add a switch to allow specified Javascript to be executed automatically when the Selense plays back – to allow a check for a "render complete" flag that we have in our Javascript.

Automated Testing, Selenium, Selenium IDE, Selenium RC, Software Development Tips, Software Testing

Functionality Verification, Regression, Stability, and Load Performance Testing of Ajax Web Sites

1 Comment

As part of stuff that I’m working on, I’m surveying various tools to test new web applications that my group is working on. There are some interesting open source tools that can do some amazing things and I’m collecting a few of the links to docs and videos I’ve seen as part of my survey. I’ll update this post with info as I find it.

We (our customers) currently live in the IE 7/Firefox world with .NET and that is the focus of the tools I’m surveying.

For this effort, I define these testing types as:

Functionality Verification – Verifying the customer specified and engineering derived requirements for an application. This also includes acceptance testing for project/contract endpoints or milestones.

Regression Testing – During development and also when releasing new builds – verify that previously implemented functionality works as expected and that there aren’t unexpected side effects from the introduction of new functionality.

Stability Testing – Verification that an application will run without leaking memory, or bogging down both the client and server side and that there aren’t low probability catastrophic events that might bring an application down with no warning and limited visibility into what happened.

Load/Performance Testing – Assess how well an application runs on given hardware so that predictions can be made for the user experience. Establish scenarios of use for the performance testing. (How many times will someone ask – how many users does you app support? – this question doesn’t make sense to me unless you define what a user is doing and the scenarios for that user’s activity).

Selenium Core, Selenium RC, Selenium IDE, Selenium Grid

Note – Before I started this effort I did not realize how invested in Selenium Google is. Looking at the Google videos on YouTube made me realize that Google has hired the initial Selenium developer and is funding internal efforts to expand Selenium. Our use of Selenium can piggyback on this significant effort from Google.

This set of tools seems to be very popular and we’ve looked at using this combination on our app. Selenium in general supports all browsers since the actual operation is a core of Javascript. The IDE portion is enticing – in that it records (from Firefox) a user’s verbs on a web user interface. These actions are saved in "Selenese" – basically an html table that contains the actions done and also can include verification of things found in the resulting HTML coming back from the web server. These test can be placed in a menu of tests to run or can be run from an app using Selenium-RC – a java based testing exec service.

One drawback is that Ajax makes testing an app difficult since there isn’t a standard way to tell when the browser is done rendering your HTML that is shown in the browser. This makes Selenium-IDE difficult to use for recording a test – since you can’t instantly run the test again without a bunch of errors saying that UI elements are not present to be looked at or acted upon (at least for a while). Also – playing games with keystrokes are difficult to record from what the IDE records.  You have to substitute recorded "typing" events in the Selenese so that your app is properly driven – creating Javascript events as normal.

Here are some things I’ve found that are useful:

Selenium: The in-browser acceptance testing tool

Jason Huggins at Google London Test Automation Conference (LTAC) giving a presentation of Selenium’s various components http://www.youtube.com/watch?v=78mts_sKNGs&NR=1 A build/verification environment is demo’d  using various OS’s. A brief overview of Selenium-IDE and its relationship to Selenium-RC – in particular recording a test in Selenese and then automatically converting to one of various languages – Java, Ruby, Python,  C#.

Selenium User Meetup 2008: Lightning Talks, March 4 2008

YouTube Video

Google is funding Selenium development internally to support their Selenium test grid (Mark Striebeck – Google Test Manager).
Selenium Grid is new – supports multiple parallel test machines. End to end web testing is slow because of the comm between clients, web servers, and database. The presentation demo’d 20 Selenium tests running at one time. Any test that works currently for Selenium RC will work in the grid. Selenium Grid is available for download now.

Jennifer Bevan of Google shared experiences of the Selenium Grid work at Google. They have 51K tests per day on their Selenium Farm. Selenium has been adopted as the primary technology for functional testing of web applications within Google. The grid can error out due to Selenium RC bugs that effectively create a DOS on the backend test servers. There has been in house Google work to create utility functions to help out.

Google is feeding back updates to the open source Selenium group.

Selenium 1.0 is coming out – RC, Core, Grid and IDE together.

Selenium 1.1 – IDE enhanced to obey remote control instruction. Coming in a couple of months.

Selenium 2.0 – is going to leverage WebDriver.

GTAC 2007: Simon Stewart – Web Driver

Simon Steward presentation on the Google Channel. http://www.youtube.com/watch?v=tGu1ud7hk5I

Paraphased from the presentation:

"Speed is important"

"Driving a real browser in memory is just going to be slow".
Web Driver is very similar to Selenium RC and uses Java to create testing objects. A big difference from Selenium RC is that Javascript cannot be driven. WD uses direct API’s into the web browser as opposed to Javascript interfaces. This makes WD more efficient than Selenium RC. However, the lack of a Javascript interface makes testing for Ajax completions very difficult. We really need to know when the render is complete and our app has a Javascript flag to check for render complete. WD specifically does not have any mechanism to test for a render complete – you need to delay to handle these issues.  This is really a killer for our app.

FAQ for webdriver: http://code.google.com/p/webdriver/wiki/FrequentlyAskedQuestions

GTAC 2007: Huggins & Stewart – Selenium-RC Vs WebDriver

Huggins & Stewart – Selenium-RC Vs WebDriver at The 2nd Annual Google Test Automation Conference (GTAC) in our New York office on August 23 and 24, 2007 http://www.youtube.com/watch?v=Vlz-WmcrBL8

Using open source tools for performance testing

Goranka Bjedov – Using open source tools for performance testing at Google London Test Automation Conference (LTAC) http://www.youtube.com/watch?v=KQ-wm3rW8_c&feature=related
Interesting point "what is a benchmark test – try to find out what customers’s are doing with the app in the real world, extract some subset of operations  that reasonably reflect the system under real operating conditions and then every time code changes, subject the system to the test under load". Goranka suggest that engineering time be put into creating a benchmark test within the scripting language of the load tool of choice.

This quote puts our problems in perspective "At Google, at any point in time there are a thousand  machines failing that need to come back in a reasonable amount of time". "Our systems are designed so that nobody should be able to see that".
About performance testing – "can our systems support what I think our customers would want".

Google’s requirement for user interface response time on search is that results are returned in less than a second and typically are in the ten’s of milliseconds.
When tests are monitored, a single resource will eventually be the bottleneck as simulated load is increased. Google recommended on Unix platforms, that the best customer experience is received when the limiting resource is not pushed above 80% of a maximum value.  Google speculated that Microsoft’s limits are typically 60%. A problem for test engineers is that marketing groups will want the numbers on throughput when the limiting resource is 100%.

Goggle suggested tools:
JMeter Java Based
Grinder 
OpenSta (Windows based)

General resources for open source testing:
http://opensourcetesting.org
http://testingreflections.org

Further reflections on this presentation from the following blog:
http://robertbaillie.blogspot.com/2006_09_01_robertbaillie_archive.html
Goranka Bjedov also posted an article about performance testing here:
http://googletesting.blogspot.com/2007/10/performance-testing.html

Software Development Tips

Oracle 10.2 Fails to Startup with Windows Server 2003 After a Reboot

Leave a comment

I spent all Friday trying to debug a problem with Oracle 10.2 starting up. I use Microsoft Virtual PC 2007 to virtualize Windows Server 2003, Oracle, and IIS 6 and I could not get Oracle to come up clean after a reboot of my server. The symptom – connect as SYS (sysdba) and the instance reports being idle.

Here’s what shows up in the oradim.log:

Mon Mar 31 10:38:58 2008
C:\oracle\product\10.2.0\db_1\bin\oradim.exe -startup -sid josaims -usrpwd * -log oradim.log -nocheck 0
Mon Mar 31 10:38:58 2008
ORA-12560: TNS:protocol adapter error

 What threw me for a loop was the ORA-12560 error pointing at listener or low level protocol errors. The solution for me turned out to be a fortunate blog post: Oracle Service Doesn’t Autostart The Instance in MS SERVER 2003R2?

Note – since I initially posted this, I discovered that this does not start the batch file at boot – it actually starts it when a user logs in. 

I’m still using the batch file, but I’m now using a utility from the Windows 2003 Resource Kit:

How to Run a Batch File Before Logging on to Your Computer

I needed to create a cmd interpreter batch file to start up the instance manually as follows:

@echo off
sleep 60
set ORACLE_SID=josaims
echo set echo on > %0.tmp
echo connect sys/password as sysdba; >> %0.tmp
echo startup; >> %0.tmp
echo exit; >> %0.tmp
sqlplus /nolog @%0.tmp
del %0.tmp

This file was placed on my Server 2003 file system and pointed to in the registry at the following key:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run

Now Oracle starts the instance on a reboot. 

What is going on here? My guess is that the Oracle instance service is starting up when networking hasn’t been fully configured in Windows Server 2003. The TNS protocol adapter error is the result since it isn’t a permanent error once the server responds to command line interpreter commands.