Open Testware Reviews
InstallWatch
Copyright 2003 by Tejas Software Consulting
- All rights reserved.
Contents
Overview -- Maturity
-- Project activity -- Platforms -- Support -- Documentation -- Installation
-- Implementation -- Performance -- Similar
tools -- Limitations -- Observations -- Appendix A:
All Files report after OpenSTA install -- Appendix
B: INI files report for OpenSTA install -- Appendix
C: Registry report after OpenSTA install
Overview
Reviewed: 2003-May-30
Version reviewed: 2.5c, 2000-Aug-25
Maintainer: Epsilon Squared,
Inc.
URL: http://www.epsilonsquared.com/
Testingfaqs.org category: Test
Implementation Tools
License: custom, binary-only
User interface: GUI, undocumented
command line
InstallWatch is a Windows-based filesystem comparator tool, able to compare
the contents of your filesystem and system registry between two points in
time. It's probably most often used to tell the user what changes a software
installation inflicts on a system, but testers can find several other uses
for the tool. For example, you can find out exactly what your software's
installation process does to your system, and compare this to the expected
behavior of the installer. You can verify that the uninstall process removes
everything it's supposed to, and no more. And for any kind of testing, you
can check the "background" -- see if your software introduced unexpected changes
on the disk during the course of testing.
I encountered a fairly long list of issues while using the tool, though
it's still useful despite the flaws.
Here's what it looks like after you've done a before-and-after analysis:
Maturity
4 - Beta (on a scale of 1-5)
The main issue preventing a production quality rating is the gaps in the
on-line help, which serves as the only documentation. There are also several
loose ends that need to be cleaned up.
Project activity
2 - Inactive (on a scale of 1-5)
As far as I can tell from externally reported information, the last
InstallWatch release was in August 2000. The latest copyright date is 1999
or 2000, depending on where in the product that you look. There are numerous
freeware catalogs that list the tool, and a few sites where users describe
their experiences. There is a mailing list dedicated to InstallWatch, but
it's largely inactive.
Further updates to the tool may be coming, but I don't have much confidence
that there will be another release any time soon.
Platforms
The target platforms for the tool are not mentioned in the internal
documentation. An external source says it runs on WIndows 95, 98, NT, and
2000. I ran it successfully on Windows 98, NT 4.0, and 2000, though the
installation on Windows 98 didn't go very smoothly.
It's worth trying on later Windows releases to see if it's forward-compatible.
Support
There is a mailing
list for InstallWatch with fifty subscribers, but practically no activity
in the last few years. The InstallRite mailing list
does have a bit of traffic and more subscribers, including responses from
the tool's author (see the Similar tools section).
There is no public bug tracking system or configuration management for
InstallWatch. Epsilon Squared is not marketing commercial support for InstallWatch,
but it couldn't hurt to ask the tool's author, Gavin Stark, if he would take
your money in exchange for support. He recently stated that he is accepting
donations and that a ramp-up in interest would encourage him to accelerate
tentative plans to release an updated version commercially.
Documentation
The only documentation for InstallWatch is the on-line help, which is
fairly thorough, though flawed in several ways. The screen shots and the
text seem to apply to a previous version of the tool, though they're close
enough to still be useful. One section of the help file implies that by
default, no files are excluded from the snapshot, but a third-party paper
and my experience indicates otherwise, and I noticed that it didn't look
at the Recycle Bin. There is a command line interface for InstallWatch,
but it is not documented at all in the on-line help (see the paper reference in the last paragraph of this section).
There is a contextual help feature - you can click the Help icon and
then click on an area of the application for help. However, in many cases
I get a "The topic does not exist" error. I also encountered this error
once within the on-line help. See the Limitations sections for details.
There is also a "Tour" page within the on-line help that's readily accessible
from several places in the application. The tour gives a nice overview of
how to use the tool.
I found several other InstallWatch reviews while googling around the Internet,
some of them written when it was a commercial tool:
It's interesting to see the different uses that people have for this
tool. The review from sacpcug.org talks about using InstallWatch to help
uninstall software (he said it didn't work well for that purpose). And one
writer described how to use InstallWatch as a security tool in "A poor-man
Tripwire-like system on Windows 9x/NT", which also includes several
insights from the tool's author that can't be found in the documentation.
Installation
InstallWatch installs from a Windows executable that weighs in at about
three and half megabytes. The "FTP Download" link on the Epsilon Squared
web site is not functional, but the "HTTP Download" link works. No reboot
is required after installation.
I did have a bit of trouble installing the tool on Windows 98 SE. On
the first attempt, the installer had a fatal error. This is perhaps a routine
scenario for Windows 98 users. I did not get that error again after a reboot,
but there was a problem setting up the uninstaller. I was not able to uninstall
InstallWatch on my Windows 98 machine, which makes for wonderful irony.
I also had trouble with interactions between InstallWatch and Netscape
7.02 on Windows NT. I couldn't get Netscape to download the install file,
though Opera handled it just fine. Also, the "Updates" button on any platform.
With Netscape as my default browser, it brings up the most recently used
Netscape window, even it it's not a Communicator window. I believe this is
a long-standing Netscape bug. But that's okay, because the URL that the Updates
feature uses is not valid anyway.
The "compact" installation option is so compact that even the main InstallWatch
executable is not installed. Don't use that option.
At the end of the installation, there is an option to join Epsilon Squared's
mailing list. This feature doesn't seem to do anything on my Windows 2000
system. On Windows 98 it starts up a message that you can send with your
default mail program.
Implementation
InstallWatch is distributed with a custom binary-only license, so there
are no implementation details to report. The source code is not available.
Performance
InstallWatch's performance is a factor of how large your registry is
and how much data is on your hard drives, and also the particular configuration
options that you choose. Checking only the registry is fairly quick, and
mostly dependent on your hard drive speed. Checking the files on the hard
drive adds more time. If you want to enable more advanced file checking to
see if a program changed the contents of a file without changing the modification
time, that'll cost you more time, and it'll bind your performance more to
cpu speed. I'm not sure what the file version info is on Windows, but I
presume it's stored somewhere in the filesystem metadata. The most difficult
check is the CRC, which reads all the data on the sections of the hard drive
that you specify in order to check that the contents didn't change.
The table below shows times in minutes and seconds from four different
runs of InstallWatch, each of which consisted of taking a snapshot, making
no deliberate changes to the system, and then running the analysis looking
for changes. I recorded these times on a slow laptop with about six and a
half gigabytes of data in about 80,000 files to check, and 107,000 registry
keys.
configuration options
|
snapshot time (mm:ss)
|
analysis time (mm:ss)
|
registry only
|
1:22
|
1:04
|
registry + file modification times
|
3:57
|
1:27
|
all of the above + file version info
|
7:53
|
8:43
|
all of the above + CRC info
|
66:48
|
63:41
|
Because of the extra time required, you probably won't want to do the
file version check or the CRC check on a regular basis unless you suspect
that something is deliberately resetting file modification times to hide
file content changes.
Similar tools
InstallWatch is often mentioned in the same breath as InCtrl5. In
fact, because InCtrl5 seems to have more of a following, I was originally
going to review it instead of InstallWatch. But PC Magazine recently
started charging a small subscription fee to download the programs that
are on their site, so that put InCtrl5 out of the freeware space. My impression
after playing with InCtrl5 a bit is that it offers a feature set very similar
to InstallWatch.
Epsilon Squared, the distributor of InstallWatch, also offers InstallRite
for free. The web site says that InstallRite lets you do "application cloning,"
in addition to all of the things that InstallWatch does, so you can clone
a software installation without going through the installation process.
I probably would have reviewed InstallRite instead if I had taken a closer
look ahead of time, especially since InstallRite has more users on its mailing
list.
A similar free Windows tool I ran across during the review is Total Uninstall.
The web page describes the same snapshot and compare process that InstallWatch
and InCtrl5 use, plus it claims to be able to reverse the effects of an
installation. It's available in several different languages.
A Windows & .NET Magazine article, "What's
the Difference," describes Microsoft's sysdiff utility that can be
useful for installation testing.
And finally, Linux users will be happy to hear that they have their own
Installwatch
tool, no relation to InstallWatch from Epsilon Squared. This tool has
a clever design. It monitors all dynamically linked programs by hooking
into the system libraries and recording system calls that modify the filesystem.
It doesn't work if a statically linked program modifies the filesystem.
The web site for this tool also refers to CheckInstall, which can use Installwatch
to create an RPM archive to clone the results of a "make install." Therefore,
CheckInstall also gives you the ability to uninstall when you use the RPM
file, even if the program doesn't have an uninstall feature.
Limitations
Here are the issues that I noted, a few of which are also mentioned
elsewhere in this review.
- The "Updates" button brings up a web browser and sends it to a
defunct URL. This feature does not work.
- If I use the "compact" installation option, the tool will not start.
- The documentation says that by default no folders or files are excluded
from the analysis, but the tool seems to exclude the Recycle Bin anyway.
A statement by the tool's author repeated in the "A poor-man Tripwire-like
system" paper lists several default exclusions, and they apparently can't
be turned off.
- On Windows 98 SE, when I install InstallWatch, I got an error:
"unInstaller setup failed to initialize". When I try to uninstall, I get
an error complaining about an uninst.isu file, and I can't uninstall the
software. When I copied an uninst.isu file from a Windows 2000 installation
and try to uninstall, I get an error that says I need to use an administrator
account, which doesn't make sense on Windows 98.
- The InstallWizard will not launch MSI installers. It reports falsely
that they completed. It seems to work only with self-extracting executables.
(I seem to remember InCtrl5 having the same problem.)
- I can print the summary pane that's visible when I click the top
item in the tree for an installation, but I can't export it to html or text.
The two export options are active but don't do anything.
- I can edit the summary pane after analyzing an installation (this
feature is undocumented). If I click on the font family or font size widgets,
the list of choices pops up in the wrong place, and all editing options
disappear until I give focus back to the summary pane.
- I can't print or export all of the analysis data with one
action. I have to save file, INI files, and the registry sections separately.
The "Print Full Report" feature mentioned in the help text doesn't exist.
- If I have at least one analysis in the database, a "Search"
item will appear in the tree. This appears to be a sophisticated feature
for searching for something, but all I get from it is a "Duplicate search
name" error when I try to use it. There is no documentation for this feature.
- When I launched a snapshot and analysis from the command line and
then later start the InstallWatch GUI, the analysis data does not show any
summary information at the top level.
- The option to join Epsilon Squared's mailing list during installation
didn't seem to do anything on Windows 2000. It seems to work on Windows 98,
though I'm not sure if anyone is listening on the other end.
- The comments on the screen when InstallWatch is doing its analysis
refer to "Back up file information," which is confusing. Perhaps it's referring
to the snapshot.
- When I try to export the analysis data for INI Files, there's no
default filename like there is for other files and the registry. Also, I
can't right-click on INI Files in the tree view to do the export like I can
for "All files" and the registry.
- The usage of the "Wait a while... I'm not finished yet" button is
confusing. Processing seems to be going on in the background somehow. Also,
the list that the button pops up includes a misspelling - "Indefinate."
Observations
Some tools start out as freeware and then go commercial. InstallWatch started
as a commercial tool and is now available as freeware. I haven't gotten
a solid picture of the history of the tool, but I'm very glad that the author,
Gavin Stark, decided to release it as freeware when Epsilon Squared stopped
marketing InstallWatch commercially. There are quite a few loose ends that
seem to be the result of the transition to freeware status, all of them
minor or fairly easy to work around. I didn't have any major frustrations
with using the tool, and most users won't be looking for trouble on purpose
like I was.
InstallWatch is basically a convenient Windows-based mechanism for comparing
the contents of your disk between two points in time. Two special cases where
the tool parses the contents of a file to give you more details about the
changes are the registry, and the registry's older cousins, INI files. It's
very interesting to see the changes that occur in the registry during the
normal course of using your computer, though it can be frustrating trying
to figure out which of the changes were initiated by the software you're testing
and which ones were initiated directly by something else.
Typically, you would use InstallWatch to report exactly what the installation
process for a software package does to your system. But there are many other
possible uses for the tool, such as the security aspect mentioned earlier,
or you could run it before and after a test run to see if the software you're
testing makes unexpected changes to the system.
Unix users can implement the same filesystem comparison mechanism with
a few dozen lines of script code. Or with something more clever like the
Linux Installwatch program, the performance would be much faster, since
changes are tracked at the source rather than exhaustively analyzing the
hard disks.
InstallWatch doesn't go as far as being able to undo an installation, though
it can save the added or both added and modified registry keys. It also
doesn't make it easy to test the uninstallation process, because after it
does its comparison, it erases the snapshot. It would be nice if it could
save a new snapshot when it does an analysis, so you could take a snapshot,
install something, do the analysis, then uninstall the software and do another
analysis.
You can start a snapshot three different ways. You can click "Install"
to start the InstallWizard, which will walk you through configuration, taking
the snapshot, doing an install, and the analysis. It would be nice if it
asked for all necessary information before taking the snapshot, so it could
do all the time-consuming tasks without interaction. I did not use the InstallWizard
very often, especially since it doesn't work with MSI files.
You can launch a comparison by configuring InstallWatch to detect when
an installation is starting. I'm not sure how that works, but it dutifully
detected my daughter installing the "Dora the Explorer Backpack Adventure"
- it took the snapshot, launched the interactive installation process, and
then did the analysis after the installation completed, frustrating my 4-year-old
all the while, but giving me a nice real-world test.
And third, the mechanism I used most often - you can ask InstallWatch manually
to take a snapshot at any time. Then do whatever you want on the system,
and when you're done, kick off the "Analyze" phase to generate the comparison.
This is the mechanism that opens up the tool to a number of different uses,
and is the only way to check an uninstallation process.
I can't help but throw in a little folklore. While I was searching for
background information about InstallWatch, I came across the Jargon File
definition for "epsilon squared." It means "A quantity
even smaller than epsilon, as small in comparison
to epsilon as epsilon is to something normal; completely negligible."
Hmmm. Gavin Stark tells me, "We chose this name since in the industry we
are small fries compared to the big players such as Microsoft or Symantec,
etc."
If you're a Windows user and want a fairly easy-to-use brute force filesystem
comparison tool, InstallWatch is worth a look.
Appendix A: All Files report after OpenSTA install
Here's the beginning of a very
long file change report from InstallWatch, after installing OpenSTA 1.4.2.
The report is exported to html format from InstallWatch.
OpenSTA 1.4.2 Release - All Files
| FileName |
Size Before |
Size After |
Attrib Before |
Attrib After |
Date Before |
Date After |
Version Before |
Version After |
CRC Before |
CRC After |
| C:\Documents and Settings\All
Users\Start Menu\Programs\OpenSTA |
|
1KB |
|
D |
|
|
|
|
|
|
| C:\Documents and Settings\All
Users\Start Menu\Programs\OpenSTA\OpenSTA Commander.lnk |
|
3KB |
|
A |
|
5/20/2003 11:51:29 AM |
|
|
|
|
| C:\Documents and Settings\All
Users\Start Menu\Programs\OpenSTA\OpenSTA NameServer.lnk |
|
3KB |
|
A |
|
5/20/2003 11:51:30 AM |
|
|
|
|
| C:\Documents and Settings\All
Users\Start Menu\Programs\OpenSTA\Web Relay Daemon.lnk |
|
3KB |
|
A |
|
5/20/2003 11:51:30 AM |
|
|
|
|
| C:\Documents and Settings\All
Users\Start Menu\Programs\Startup\OpenSTA NameServer.lnk |
|
3KB |
|
A |
|
5/20/2003 11:56:20 AM |
|
|
|
|
| C:\Program Files\OpenSTA\BaseUI |
|
1KB |
|
D |
|
|
|
|
|
|
| C:\Program Files\OpenSTA\BaseUI\NTPerfPlugin.exe |
|
123KB |
|
A |
|
5/2/2003 12:02:18 AM |
|
|
|
|
| C:\Program Files\OpenSTA\BaseUI\OpenSTACommander.chm |
|
870KB |
|
A |
|
4/29/2003 3:32:02 AM |
|
|
|
|
| C:\Program Files\OpenSTA\BaseUI\OSCommander.exe |
|
308KB |
|
A |
|
5/2/2003 12:00:42 AM |
|
|
|
|
...
|
|
|
|
|
|
|
|
|
|
|
Appendix B: INI files report for OpenSTA install
This is the INI files section for the OpenSTA install.
| INI File Name |
Section |
Item |
Data Before |
Data After |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
NUMCAT |
|
11 |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT1 |
|
system,system |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT2 |
|
interfaces,interfaces |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT3 |
|
at,at |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT4 |
|
ip,ip |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT5 |
|
icmp,icmp |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT6 |
|
tcp,tcp |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT7 |
|
udp,udp |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT8 |
|
egp,egp |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT9 |
|
dot3,dot3 |
| C:\Program Files\OpenSTA\BaseUI\snmpcat.ini |
CATEGORY |
CAT10 |
|
snmp,snmp |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
FILES |
TraceLevel |
|
0 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
SOCKET |
MaxSocketDataBuffersCount |
|
64000 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
SOCKET |
SocketDataBuffersGrowingCount |
|
200 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
SOCKET |
MaxSSLConcurrentReq |
|
8000 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
SOCKET |
SSLGrowingBuffCount |
|
100 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
TEST |
BrowserParallelism |
|
4 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
TEST |
InitialVirtualUserCount |
|
10 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
TEST |
VirtualUserGrowBy |
|
10 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
THREAD POOL |
ThreadPoolSize |
|
0 |
| C:\Program Files\OpenSTA\Engines\TestExecuter_web.ini |
THREAD POOL |
ThreadPoolConcurrentThreads |
|
0 |
Appendix C: Registry report after OpenSTA install
Here's the beginning of a very long registry report from InstallWatch.
OpenSTA 1.4.2 Release - Registry
| Key |
Value |
Data Before |
Data After |
| HKEY_CLASSES_ROOT\.HTP |
|
|
|
| HKEY_CLASSES_ROOT\.HTP\DEFAULTICON |
|
|
|
| HKEY_CLASSES_ROOT\.HTP\DEFAULTICON |
@ |
|
"C:\Program Files\OpenSTA\Engines\Web\Modeller\TModeller_web.exe" |
| HKEY_CLASSES_ROOT\.HTP\SHELL |
|
|
|
| HKEY_CLASSES_ROOT\.HTP\SHELL\EDIT |
|
|
|
| HKEY_CLASSES_ROOT\.HTP\SHELL\EDIT\COMMAND |
|
|
|
| HKEY_CLASSES_ROOT\.HTP\SHELL\EDIT\COMMAND |
(Default) |
|
"C:\Program Files\OpenSTA\Engines\Web\Modeller\TModeller_web.exe
"%1"" |
| HKEY_CLASSES_ROOT\.SMP |
|
|
|
| HKEY_CLASSES_ROOT\.SMP |
@ |
|
"SNMPPlugin.Document" |
| HKEY_CLASSES_ROOT\.SMP\SNMPPlugin.Document |
|
|
|
| ... |
|
|
|