Open Testware Reviews

OpenSTA (Open System Testing Architecture)

Contents

Overview

Maturity

Project activity

Platforms

OpenSTA runs on Windows NT SP5, Windows 2000, and Windows XP. NT users need Windows Installer for Windows NT 1.1. The tool needs version 2.5 or later of the Microsoft Data Access Components. The Users Guide calls for a minimum of a Pentium 200 processor, 80MB RAM, and 20MB hard disk space. OpenSTA supports HTTP 1.0 and 1.1, and HTTPS (SSL).

The Script Modeler directly supports using Netscape Navigator 4.7 and Internet Explorer 4, 5 and 6 for script recording, but with some manual setup you can use any browser that supports proxies. To use the Web Relay Daemon, you'll need a web server on your test machine that supports ISAPI.

Support

• opensta-announce - OpenSTA project update announcements (currently inactive)
• opensta-users - OpenSTA users discussion and support (an active list)
• opensta-devel - OpenSTA developer discussions (a fairly active list)

• Bugs (69 open / 258 total)
• Feature Requests (16 open / 26 total)

• etest associates, based in the UK, has the only solid offer for technical support for OpenSTA. They tell me that they plan to offer on-site training soon. Etest associates seems to be respected by the OpenSTA community.
• RTTS, based in the U.S., offers consulting services using OpenSTA. I confirmed this with Bill Hayduk, Director of Professional Services for RTTS.
• Zebsys, based in the UK, offers training and consulting for OpenSTA.
• Abstrakt Gmbh, based in Germany, reportedly provides consulting services associated with the OpenSTA tool.
• Quaver Computing, based in the UK, offers training and consulting for OpenSTA, some or all of which is resold from etest associates.
• Daniel Sutcliffe, the current lead for the OpenSTA SourceForge project, tells me that he's interested in offering OpenSTA services through his company tcNOW.com. He's based in the U.S.
  

Documentation

• OpenSTA Getting Started Guide for OpenSTA 1.3
• OpenSTA User's Guide
• Script Control Language Reference Guide, also available in French
  

Installation

Implementation

OpenSTA has a substantial source code base. I looked at the source package for version 1.4.1, available as a zip file via ftp. It's about 300,000 lines of non-comment source code, mostly C++. There are about 280,000 lines of comments as well, though I'm told that most of the comments are file headers and the code is actually not very well documented. The OpenSTA Build Instructions give some information about the structure of the code, though it looks like there would be a large learning curve in understanding any large fraction of the code simply because it's such a large source base.

OpenSTA sources are available for download separately from the binary installation package - the source is almost six megabytes. There are five additional components not developed by the OpenSTA project that must be downloaded from various web sites in order to build the tool. I did not try to build OpenSTA from the source because it requires the proprietary Microsoft Visual C++ compiler, which I don't have. 

I didn't find any test cases for the tool. There does appear to be some hooks for memory allocation debugging in part of the code, though the hooks probably aren't being used.

If you want to build OpenSTA from the source, it is recommended to get the latest sources from the CVS server rather than relying on the 1.4.1 source package.

Performance

Similar tools

Limitations

Observations

Appendix A: Sample SCL code

!Browser:IE5
!Date : 3/17/03
Environment
	Description ""
	Mode		HTTP
	Wait		UNIT MILLISECONDS

Definitions
	! Standard Defines
	Include			"RESPONSE_CODES.INC"
	Include			"GLOBAL_VARIABLES.INC"

	CHARACTER*512	USER_AGENT
	CHARACTER*256	MESSAGE

	Integer			USE_PAGE_TIMERS
	Timer			T_FINDBYNAME
	CHARACTER*1024	cookie_2_0
	CHARACTER*1024	cookie_3_0
	CHARACTER*1024	cookie_4_0

	CONSTANT		DEFAULT_HEADERS = "Host: 192.168.1.113:8080^J"	&
		"User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0)"
		
    CONSTANT		HOST = "thinkpad:8080"
    CHARACTER*1024	URLBUF

Code
	!Read in the default browser user agent field
	Entry[USER_AGENT,USE_PAGE_TIMERS]

	Start Timer T_FINDBYNAME
	SET URLBUF = "http://" + HOST + "/cgi-bin/findpres.plx HTTP/1.0"
	PRIMARY GET URI URLBUF ON 1			&
		HEADER DEFAULT_HEADERS			&
		,WITH {"Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms" &
				"-excel, application/vnd.ms-powerpoint, application/msword, */*",			&
			"Accept-Language: en-us",			&
			"Connection: Keep-Alive"}

	DISCONNECT FROM 1

	WAIT 9404

    SET URLBUF = "http://" + HOST + "/cgi-bin/findpres.plx HTTP/1.0"
	PRIMARY POST URI URLBUF ON 2			&
		HEADER DEFAULT_HEADERS			&
		,WITH {"Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms" &
				"-excel, application/vnd.ms-powerpoint, application/msword, */*",			&
			"Referer: http://192.168.1.113:8080/cgi-bin/findpres.plx",			&
			"Accept-Language: en-us",			&
			"Content-Type: application/x-www-form-urlencoded",			&
			"Connection: Keep-Alive",			&
			"Content-Length: 18",			&
			"Pragma: no-cache"}			&
		,BODY "loginid=a&passwd=a"

	Load Response_Info Header on 2		&
		Into cookie_2_0	&
		,WITH "Set-Cookie,findpresid"

	DISCONNECT FROM 2

	WAIT 6990

    SET URLBUF = "http://" + HOST + "/cgi-bin/findpres.plx HTTP/1.0"
	PRIMARY POST URI URLBUF ON 3			&
		HEADER DEFAULT_HEADERS			&
		,WITH {"Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms" &
				"-excel, application/vnd.ms-powerpoint, application/msword, */*",			&
			"Referer: http://192.168.1.113:8080/cgi-bin/findpres.plx",			&
			"Accept-Language: en-us",			&
			"Content-Type: application/x-www-form-urlencoded",			&
			"Connection: Keep-Alive",			&
			"Content-Length: 23",			&
			"Pragma: no-cache",			&
			"Cookie: "+cookie_2_0}			&
		,BODY "np=washington&pp=&re=&dz=ok"

	Load Response_Info Header on 3		&
		Into cookie_3_0	&
		,WITH "Set-Cookie,findpresid"

	DISCONNECT FROM 3

	WAIT 4817

    SET URLBUF = "http://" + HOST + "/cgi-bin/findpres.plx?loginid=a&logout=ok HTTP/1.0"
	PRIMARY GET URI URLBUF ON 4			&
		HEADER DEFAULT_HEADERS			&
		,WITH {"Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms" &
				"-excel, application/vnd.ms-powerpoint, application/msword, */*",			&
			"Referer: http://192.168.1.113:8080/cgi-bin/findpres.plx",			&
			"Accept-Language: en-us",			&
			"Connection: Keep-Alive",			&
			"Cookie: "+cookie_3_0}

	Load Response_Info Header on 4		&
		Into cookie_4_0	&
		,WITH "Set-Cookie,findpresid"

	DISCONNECT FROM 4

	SYNCHRONIZE REQUESTS

	End Timer T_FINDBYNAME
Exit

ERR_LABEL:

	If (MESSAGE <> "") Then
		Report MESSAGE
	Endif

Exit

Appendix B: Replay log

!Begin replay
1-1 :[32]:START TIMER: T_FINDBYNAME
1-1 :[33]:SET VARIABLE: URLBUF to value http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0,Index 
1-1 :[34]:REQUEST(1): GET http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0
1-1 :[41]:DISCONNECT(1)
1-1 :[41]:Connection 1: receiving results with status 200
1-1 :[43]:WAIT : Delay: 9404
1-1 :[45]:SET VARIABLE: URLBUF to value http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0,Index 
1-1 :[46]:REQUEST(2): POST http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0
1-1 :[58]:Connection 2: receiving results with status 200
1-1 :[58]:HTTPRESPONSE(2): Parsing header contents. Name: COOKIE_2_0 Identity: Set-Cookie,findpresid
1-1 :[58]:HTTPRESPONSE(2): Setting COOKIE_2_0 to 'findpresid=a,1048775818'
1-1 :[62]:DISCONNECT(2)
1-1 :[64]:WAIT : Delay: 6990
1-1 :[66]:SET VARIABLE: URLBUF to value http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0,Index 
1-1 :[67]:REQUEST(3): POST http://thinkpad:8080/cgi-bin/findpres.plx HTTP/1.0
1-1 :[80]:Connection 3: receiving results with status 200
1-1 :[80]:HTTPRESPONSE(3): Parsing header contents. Name: COOKIE_3_0 Identity: Set-Cookie,findpresid
1-1 :[80]:HTTPRESPONSE(3): Setting COOKIE_3_0 to 'findpresid=a,1048775826'
1-1 :[84]:DISCONNECT(3)
1-1 :[86]:WAIT : Delay: 4817
1-1 :[88]:SET VARIABLE: URLBUF to value http://thinkpad:8080/cgi-bin/findpres.plx?loginid=a&logout=ok HTTP/1.0,Index 
1-1 :[89]:REQUEST(4): GET http://thinkpad:8080/cgi-bin/findpres.plx?loginid=a&logout=ok HTTP/1.0
1-1 :[98]:Connection 4: receiving results with status 200
1-1 :[98]:HTTPRESPONSE(4): Parsing header contents. Name: COOKIE_4_0 Identity: Set-Cookie,findpresid
1-1 :[98]:HTTPRESPONSE(4): Setting COOKIE_4_0 to 'findpresid=,0'
1-1 :[102]:DISCONNECT(4)
1-1 :[104]:SYNCHRONIZE
1-1 :[104]:All requests synchronised.
1-1 :[106]:STOP TIMER: T_FINDBYNAME
1-1 :[107]:TRcrdExit, step 0: Cancelled all requests. Now waiting for the terminating script requests to complete.
1-1 :[107]:TRcrdExit, step 1: Script requests completed. Disconnecting all clients.
1-1 :[107]:TRcrdExit, step 2: Script clients disconnected.
1-1 :[107]:TOF execution is over.
!Script Completed Successfully.