Performance testing is a general term used to describe testing activities where a simulated user load is directed at a system and measurements are gathered. It is designed to meet or exceed a set of system performance goals while maintaining a particular user load profile. It places variable load – from a minimum to a maximum – on the system. The variable load demonstrates what the system can sustain without running out of resource or having transactions perform in a less-than-optimal fashion.

Performance testing is a suite of testing that can consists of, but is not limited to, Benchmark Testing, Load Testing, Durability Testing, Volume Testing, Stress Testing, and Scalability Testing. In some cases a separate Performance Test Plan will be created to explain the details of the Performance Testing effort.

• Benchmark Testing: Benchmark testing is a performance test which subjects the system to varying workloads to measure and evaluate the performance behaviors and ability of the system to continue to function properly under these different workloads.
• Durability Testing: Durability testing is designed to determine the characteristics of a system under load conditions, over time. This is an excellent test to identify memory leaks in a system.
• Load Testing: Load testing is a performance test which subjects the system to varying workloads to measure and evaluate the performance behaviors and ability of the system to continue to function properly under these different workloads. The goal of load testing is to determine and ensure that the system functions properly beyond the expected maximum workload. Additionally, load testing evaluates the performance characteristics (response times, transaction rates, and other time sensitive issues).
• Scalability Testing: Scalability Testing determines the ability of a system to expand as the usage increases. It measures individual components of the system to determine their performance characteristics under increasing user loads. It also examines inter-component and multiple component behavior.
• Stress Testing: Stress Testing determines the ability of the application to perform under low and/or excessive loads to ensure that critical information and services are available when and how the end-user expects it. In addition, it is also a way to identify and document conditions under which the system fails to continue functioning properly.
• Volume Testing: Volume Testing subjects the target-of-test to large amounts of data to determine if limits are reached that cause the software to fail. Volume testing also identifies the continuous maximum load or volume the target-of-test can handle for a given period.

NOTE: Transactions above refer to “logical business transactions.” These transactions are defined as specific functions that an end user of the system is expected to perform using the application, such as add or modify a given contract.

Below are some open source/free tools that can help you with performance testing the application under test (AUT).

Please keep in mind that every AUT is different so the tool you pick for one application may not be the same tool that you pick for another. My advice would be to pick a tool that can meet most of your current and near future needs.

• Allmon: A generic system collecting and storing metrics used for performance and availability monitoring
• Apache JMeter: A 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. Apache JMeter may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load. Requirement: Solaris, Linux, Windows (98, NT, 2000). JDK1.4 (or higher).
• benerator: A framework for creating realistic and valid high-volume test data, used for (unit/integration/load) testing and showcase setup. Metadata constraints are imported from systems and/or configuration files. Data can be imported from and exported to files and systems, anonymized or generated from scratch. Domain packages provide reusable generators for creating domain-specific data as names and addresses internationalizable in language and region. It is strongly customizable with plugins and configuration options. Requirement: Platform Independent
• CLIF is a Load Injection Framework: A modular and flexible distributed load testing platform. It may address any target system that is reachable from a Java program (HTTP, DNS, TCP/IP…) CLIF provides 3 user interfaces (Swing or Eclipse GUI, command line) to deploy, control and monitor a set of distributed load injectors and resource consumption probes (CPU, memory…) An Eclipse wizard helps programming support for new protocols. Load scenarios are defined through XML-editing, using a GUI, or using a capture tool. The scenario execution engine allows the execution of up to millions of virtual users per load injector. Requirement: Java 1.5 or greater, with enhanced support for Linux, Windows XP, MacOSX/PPC
• curl-loader: A C-written web application testing and load generating tool. The goal of the project is to provide a powerful open-source alternative to Spirent Avalanche and IXIA IxLoad. The loader uses real HTTP, FTP and TLS/SSL protocol stacks, simulating tens of thousand and hundred users/clients each with own IP-address. The tool supports user authentication, login and a range of statistics. Requirement: linux
• Database Opensource Test Suite (DOTS): A set of test cases designed for the purpose of stress-testing database server systems in order to measure database server performance and reliability. Requirement: Linux, POSIX
• DBMonster: Application to generate random data for testing SQL database driven applications under heavy load. Requirement: OS Independent

• Deluge: Simulates multiple user types and counts. Includes proxy server for recording playback scripts, and log evaluator for generating result statistics. Note: this tool is no longer under active development although it is still available on Sourceforge. Requirement: OS independent
• Dieseltest: Contains the high-end features common to packages costing $50,000 or more. Dieseltest is a Windows application that simulates hundreds or thousands of users hitting a website. To run a load test, you first create a test script using our script editor. The script contains all of the requests that a real-world user would make of a website. You then load the script and run the test. The system will show you real-time results while the script is running, and produce a report analyzing the results at the conclusion. Requirement: Windows
• Eclipse Test & Performance Tools Platform Project (TPTP): Supplies powerful frameworks and services that allow software developers to build unique test and performance tools, both open source and commercial, that can be easily integrated with the platform and with other tools
• Faban: A facility for developing and running benchmarks, developed by Sun. It has two major components, the Faban harness and the Faban driver framework. The Faban harness is a harness to automate running of server benchmarks as well as a container to host benchmarks allowing new benchmarks to be deployed in a rapid manner. Faban provides a web interface to launch & queue runs, and extensive functionality to view, compare and graph run outputs. Requirement: OS independent; JVM 1.5 or later.
• FunkLoad: A functional and load web tester, written in Python, whose main use cases are functional and regression testing of web projects, performance testing by loading the web application and monitoring your servers, load testing to expose bugs that do not surface in cursory testing, and stress testing to overwhelm the web application resources and test the application recoverability, and writing web agents by scripting any web repetitive task, like checking if a site is alive. Requirement: OS independent – except for the monitoring which is Linux specific.
• FWPTT: Web application testing program for load testing web applications. It can record normal and AJAX requests. It has been tested on ASP.Net applications, but it should work with JSP, PHP or other. Requirement: windows
• Grinder: A Java-based load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded. Requirement: OS Independent
• Hammerhead 2: A stress testing tool designed to test out your web server and web site. It can initiate multiple connections from IP aliases and simulated numerous (256+) users at any given time. The rate at which Hammerhead 2 attempts to pound your site is fully configurable, there are numerous other options for trying to create problems with a web site (so you can fix them). Requirement:Hammerhead has been used with Linux, Solaris and FreeBSD.
• Hammerora: A load generation tool for the Oracle Database and Web Applications. Hammerora includes pre-built schema creation and load tests based on the industry standard TPC-C and TPC-H benchmarks to deploy against the Oracle database with multiple users. Hammerora also converts and replays Oracle trace files and enables Web-tier testing to build bespoke load tests for your entire Oracle application environment. Requirement: Platform Independent (Binaries for Linux and Windows)
• httperf: A tool for measuring web server performance. It provides a flexible facility for generating various HTTP workloads and for measuring server performance. The focus is not on implementing one particular benchmark but on providing a robust, high-performance tool that facilitates the construction of both micro and macro level benchmarks. The three distinguishing characteristics of httperf are its robustness, which includes the ability to generate and sustain server overload, support for the HTTP/1.1 and SSL protocols, and its extensibility. Requirement: linux (Debian package available), HP-UX, perhaps other Unix
• http_load: Runs multiple HTTP fetches in parallel, to test the throughput of a Web server. However, unlike most such test clients, it runs in a single process, to avoid bogging the client machine down. It can also be configured to do HTTPS fetches. Requirement: tbc
• IxoraRMS: A tool for gathering and visualizing monitoring data. Requirement: Windows, Unix
• JChav: A way to see the change in performance of your web application over time, by running a benchmark test for each build you produce. JChav reads all the JMeter logs from each of your runs (one per build), and produces a set of charts for each test in each run. Requirement: JMeter
• JCrawler: Stress-Testing Tool for web-applications. It comes with the crawling/exploratory feature. You can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. The load parameters (hits/sec) are configurable. Requirement: OS Independent
• Load Impact : Online load testing service from Gatorhole/loadimpact.com for load- and stress- testing of your website over the Internet; access to our distributed network of load generator nodes – server clusters with very fast connections to enable simulation of tens of thousands of users accessing your website concurrently. Free low level load tests for 1-50 simulated users; higher levels have monthly fees.
• Lobo, Continuous Tuning: A tool for performance testing and monitoring that allows you to monitor the evolution of performance along the time-line of the project. It was specially designed to be used in agile-iterative and evolutionary approaches. Requirement: Java
• MessAdmin: A light-weight and non-intrusive notification system and HttpSession administration for J2EE Web Applications, giving detailed statistics and informations on the application. It installs as a plug-in to any Java EE WebApp, and requires zero-code modification. Requirement: OS Independant
• Multi Router Traffic Grapher (MRTG): Written in perl and works on Unix/Linux as well as Windows and even Netware systems. MRTG is free software licensed under the Gnu GPL.
• NTime: Very similar to NUnit tool to perform repeatable tasks that help managers, architects, developers and testers to test an application against its performance. Requirement: Windows 98 or above, .Net framework 1.1 or 2.0
• OpenSTA: A distributed software testing architecture based on CORBA. Using OpenSTA (Open System Testing Architecture) a user can generate realistic heavy loads simulating the activity of hundreds to thousands of virtual users. OpenSTA graphs both virtual user response times and resource utilization information from all Web Servers, Application Servers, Database Servers and Operating Platforms under test, so that precise performance measurements can be gathered during load tests and analysis on these measurements can be performed. Requirement: Windows 2000, NT4 and XP
• OpenWebLoad: A tool for load testing web applications. It aims to be easy to use and providing near real-time performance measurements of the application under test. Requirement: Linux, Windows
• p-unit: Framework for unit test and performance benchmark, which was initiated by Andrew Zhang, under GPL license. p-unit supports to run the same tests with single thread or multi-threads, tracks memory and time consumption, and generates the result in the form of plain text, image or pdf file. Requirement: OS Independent
• PandoraFMS: A monitoring software. It watches your systems and applications, and allows you to know the status of any element of those systems. Pandora FMS could detect a network interface down, a defacement in your website, a memory leak in one of your server application, or the movement of any value of the NASDAQ new technology market. If you want, Pandora FMS could send out SMS message when your systems fails… or when Google’s value drop below US$ 500. Requirement: 32-bit MS Windows (NT/2000/XP), All POSIX (Linux/BSD/UNIX-like OSes), Solaris, HP-UX, IBM AIX
• Parallel-junit: Small library extensions for JUnit. Extends the junit.framework.TestSuite class by running tests in parallel, allowing more efficient test execution. Because TestResult and TestListener aren’t designed to run tests in parallel, this implementation coordinates the worker threads and reorder event callbacks so that the TestResult object receives them in an orderly manner. In addition, output to System.out and System.err are also serialized to avoid screen clutter.

• Pylot: Tool for testing performance and scalability of web services. It runs HTTP load tests, which are useful for capacity planning, benchmarking, analysis, and system tuning. Pylot generates concurrent load (HTTP Requests), verifies server responses, and produces reports with metrics. Tests suites are executed and monitored from a GUI. Requirement: Python 2.5+. required.Tested on Windows XP, Vista, Cygwin, Ubuntu, MacOS
• Raw Load Tester: This application calls the URL you select as many times as you choose and tells you how long it took the server to respond. It writes some additional runtime details to the PHP log file so you can optionally do more granular analysis afterwards. Although the server processes most of the statistics, all URL requests come from the browser. You can run as many browsers and workstations simultaneously as you want. Requirement: PHP/JavaScript
• RRDtool: High performance data logging and graphing system for time series data. Use it to write your custom monitoring shell scripts or create whole applications using its Perl, Python, Ruby, TCL or PHP bindings.
• Seagull: A multi-protocol traffic generator test tool. Primary aimed at IMS protocols, Seagull is a powerful traffic generator for functional, load, endurance, stress and performance tests for almost any kind of protocol. Currently supports Diameter, XCAP over HTTP, TCAP (GSM Camel, MAP, Win) protocols. Requirement: Linux/Unix/Win32-Cygwin
• Siege: A http regression testing and benchmarking utility. It was designed to let web developers measure the performance of their code under duress, to see how it will stand up to load on the internet. It lets the user hit a webserver with a configurable number of concurrent simulated users. Those users place the webserver “under siege.” SCOUT surveys a webserver and prepares the urls.txt file for a siege. In order to perform regression testing, siege loads URLs from a file and runs through them sequentially or randomly. Scout makes the process of populating that file easier. You should send out the scout, before you lay siege. Requirement: GNU/Linux, AIX, BSD, HP-UX and Solaris.
• SIPp: A performance testing tool for the SIP protocol. Its main features are basic SIPStone scenarios, TCP/UDP transport, customizable (xml based) scenarios, dynamic adjustement of call-rate and a comprehensive set of real-time statistics. It can also generate media (RTP) traffic for audio and video calls. Requirement: Linux/Unix/Win32-Cygwin
• SiteBlaster: A web site load and stress testing tool. It can be used to rapidly submit requests to a site. Or, it can pause a random amount of time between submissions; approximating the behavior of a user. While the testing is being performed, the pages being tested will be displayed. When testing is complete, a report is available that can be viewed or printed. Requirement: Windows Installer v 3.1 and Microsoft .Net Framework 3.5 SP1
• SLAMD: A Java-based application designed for stress testing and performance analysis of network-based applications. Requirement: Any system with Java 1.4 or higher
• Soap-Stone: Network benchmark application which can put your network under load and conduct automatic benchmark and recording activities. Requirement: OS Independent
• stress_driver: General-purpose stress test tool. Requirement: Windows NT/2000, Linux
• TestMaker: Delivers a rich environment for building and running intelligent test agents that test Web-enabled applications for scalability, functionality, and performance. It comes with a friendly graphical user environment, an object-oriented scripting language (Jython) to build intelligent test agents, an extensible library of protocol handlers (HTTP, HTTPS, SOAP, XML-RPC, SMTP, POP3, IMAP), a new agent wizard featuring an Agent Recorder to write scripts for you, a library of fully-functional sample test agents, and shell scripts to run test agents from the command line and from unit test utilities. Requirement: Java 1.4 or higher virtual machine on Windows, Linux, Solaris, and Macintosh.
• TPTEST: The purpose with TPTEST is to allow users to measure the speed of their Internet connection in a simple way. TPTEST measures the throughput speed to and from various reference servers on the Internet. The use of TPTEST may help increase the consumer/end user knowledge of how Internet services work. Requirement: MacOS/Carbon and Win32
• Tsung: A protocol-independent and can currently be used to stress HTTP, SOAP and Jabber servers (SSL is supported). It simulates complex user’s behaviour using an XML description file, reports many measurements in real time (including response times, CPU and memory usage from servers, customized transactions, etc.). HTML reports (with graphics) can be generated during the load. For HTTP, it supports 1.0 and 1.1, has a proxy mode to record sessions, supports GET and POST methods, Cookies, and Basic WWW-authentication. It has already been used to simulate thousands of virtual users. Requirement: Tested on Linux, but should work on MacOSX and Windows.
• Valgrind: Suite of tools for debugging and profiling Linux programs. With the tools that come with Valgrind, you can automatically detect many memory management and threading bugs, avoiding hours of frustrating bug-hunting, making your programs more stable. You can also perform detailed profiling, to speed up and reduce memory use of your programs. Requirement: Linux
• VisualVM: A free visual tool, originally from Sun, to monitor and troubleshoot Java applications. Runs on Sun JDK 6, but is able to monitor applications running on JDK 1.4 and higher. Utilizes various available technologies like jvmstat, JMX, the Serviceability Agent (SA), and the Attach API to get data and uses minimal overhead on monitored applications. Capabilities include: automatically detects and lists locally and remotely running Java applications; monitor application performance and memory consumption; profile application performance or analyze memory allocation; is able to save application configuration and runtime environment together with all taken thread dumps, heap dumps and profiler snaphots into a single application snapshot which can be later processed offline.
• Web Application Load Simulator: A web application load simulator. It allows you to create simulations and have those simulations run against your webserver. Requirement: JDK 1.3 or above
• WebInject: It can be used to test individual system components that have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc), and can be used as a test harness to create a suite of [HTTP level] automated functional, acceptance, and regression tests. A test harness allows you to run many test cases and collect/report your results. WebInject offers real-time results display and may also be used for monitoring system response times
• Web Polygraph: Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows. Requirement: C++ compiler
• WebLOAD: A fully functional, commercial-grade performance testing product based on WebLOAD, Radview’s flagship product that is already deployed at 1,600 sites. Available for free download and use, WebLOAD is a commercial-grade open source project with more than 250 engineering years of product development. Companies that require commercial support, additional productivity features and compatibility with third-party protocols have the option of purchasing WebLOAD Professional directly from RadView. Requirement: Windows NT/2000/XP
• Xceptance LoadTest: Load testing and regression tool from Xceptance Software Technologies, Inc for web and Java and other app load testing. Includes recording capabilities. XLT Cloud Service available. Tests implemented as JUnit 4 test cases. For web-based tests, the framework provides a (headless) browser that can emulate Internet Explorer or Firefox behaviour. Can execute client-side JavaScript in the emulated web browsers and that way it simplifies the creation of test cases for Web 2.0 applications. Platform independent due to tool being implemented in Java; test scripting in Java or Ruby. Free for up to five virtual users.
• zapwireless: A wireless throughput test tool which can be used to fully characterise the statistical performance of a wireless link.

Please comment below on your thoughts/experience on any of the tools listed above and/or if you know of any other tools that should be added.

Questions:

• Have you used any of these?
• If so, which one are you using today?
• Why did you pick the tool you are using?

Related posts:

1. 15+ Open Source Test Management Tools
2. 100+ Open Source/Free Functional Testing Tools
3. 10+ Open Source Link Checking Tools

This is a extension of the blog post “10+ Software Testing Blogs“. This blog post has 100+ software testing blogs and tweeters.

I also created the Twitter List “@jayphilips/software-testing” so you can follow all of the people listed on this post with one just click.

• Abby Fichtner
  • Visit Blog
  • Follow HackerChick on Twitter
• Advanced Brain
  • Follow AdvancedBrain on Twitter
• Advanced QTP
  • Visit Site
  • Follow AdvancedQTP on Twitter
• Agilebuddy
  • Visit Blog
  • Follow Agilebuddy on Twitter
• AgilePoint
  • Follow AgilePoint on Twitter
• Aimee Lack
  • Follow Test_Monkey on Twitter
• Alan A. Jorgensen
  • Follow SoftTest123 on Twitter
• Alan Page
  • Visit Blog
  • Follow AlanPage on Twitter
• Alan Richardson
  • Visit Blog
• Andy Tinkham
  • Follow Andy Tinkham on Twitter
• Anne-Marie Charrett
  • Visit Blog
  • Follow Charrett on Twitter
• Antony Marcano
  • Visit Blog
  • Follow AntonyMarcano on Twitter
• Arjan Kranenburg
  • Visit Blog
  • Follow ArjanKranenburg on Twitter
• Ascert Testing
  • Follow Ascert on Twitter
• AST
  • Visit Site
  • Follow AST_News on Twitter
• automatedtest
  • Follow automatedtest on Twitter
• Ayal Zylberman
  • Follow AyalZ on Twitter
• Ben Simo
  • Visit Blog
  • Follow QualityFrog on Twitter
• BJ Rollison
  • Visit Blog
• Bret Pettichord
  • Visit Blog
  • Follow BPettichord on Twitter
• Catherine Powell
  • Visit Blog
• Cem Kaner
  • Visit Blog
• Chris McMahon
  • Visit Blog
  • Follow chris_mcmahon on Twitter
• Cloud Test
  • Follow CloudTest on Twitter
• Cloud Testing
  • Follow CloudTesting on Twitter
• Corey Goldberg
  • Visit Blog
  • Follow CGoldberg on Twitter
• Daily Testing Tip
  • Visit Site
  • Follow DailyTestingTip on Twitter
• Dave Liebreich
  • Follow ATestGuy on Twitter
• David Alfaro
  • Visit Blog
  • Follow AgileNature on Twitter
• David Evans
  • Follow DavidEvans66 on Twitter
• Debasis Pradhan
  • Visit Blog
  • Follow DebasisPradhan on Twitter
• Elisabeth Hendrickson
  • Visit Blog
  • Follow TestObsessed on Twitter
• Eric Jacobson
  • Visit Blog
• Esther Derby
  • Visit Blog
  • Follow EstherDerby on Twitter
• Frank Cohen
  • Follow FCohen on Twitter
• Fred Beringer
  • Visit Blog
  • Follow FredBeringer on Twitter
• Gerald Weinberg
  • Follow JerryWeinberg on Twitter
• Glenn Halstead
  • Follow GlennHalstead on Twitter
• Go Test It
  • Visit Blog
  • Follow GoTestIt on Twitter
• Gojko Adzic
  • Visit Blog
  • Follow GojkoAdzic on Twitter
• Gustavo Terrera
  • Visit Blog (in Spanish)
  • Follow testingok on Twitter
• Guy Mason
  • Follow testingqa on Twitter
• Henrik Andersson
  • Follow HenkeAndersson on Twitter
• Jake Brake
  • Visit Blog
• James Bach
  • Visit Blog
  • Follow JamesMarcusBach on Twitter
• James Christie
  • Follow James_Christie on Twitter
• James Whittaker
  • Visit Blog
• Jay Philips
  • You are currently here
  • Follow JayPhilips on Twitter
• Jeff Fry
  • Visit Blog
  • Follow JFry on Twitter
• Jeff Sutherland
  • Visit Blog
  • Follow JeffSutherland on Twitter
• Jeroen Rosink
  • Visit Blog
  • Follow JeroenRo on Twitter
• Joel Montvelisky
  • Visit Blog
  • Follow JoelMonte on Twitter
• Joel Spolsky
  • Visit Blog
  • Follow Spolsky on Twitter
• Johan Jonasson
  • Follow JohanJonasson on Twitter
• Dr. John A. Estrella
  • Visit Blog
  • Follow JohnEstrella on Twitter
• John F
  • Visit Blog
  • Follow lifeinqa on Twitter
• Jon Bach
  • Visit Blog
  • Follow jbtestpilot on Twitter
• Jonathan Kohl
  • Visit Blog
• Joseph Ours
  • Visit Blog
  • Follow JustJoeHere on Twitter
• Justin Dessonville
  • Visit Blog
  • Follow iamdez on Twitter
• Justin Hunter
  • Follow Hexawise on Twitter
• Justin Rohrman
  • Visit Blog
  • Follow JRohrman on Twitter
• Karen Johnson
  • Visit Blog
• Karen Tobo
  • Follow KarenTobo on Twitter
• Kiran Adapathya
  • Follow Kiran_Adapathya on Twitter
• Linda Wilkinson
  • Visit Blog
• 
• Mark Crowther
  • Visit Blog
  • Follow MarkCTest on Twitter
• Mark Tomlinson
  • Visit Blog
  • Follow MTomlins on Twitter
• Marta G. Ferrero
  • Follow MartaGF on Twitter
• Martin L. Shoemaker
  • Follow UMLGuy on Twitter
• Mat Johnston
  • Follow MatJohnston on Twitter
• Matthew Heusser
  • Visit Blog
  • Visit Blog on ST&P
  • Follow MHeusser on Twitter
• Michael Bolton
  • Visit Blog
  • Follow MichaelBolton on Twitter
• Mike Cottmeyer
  • Visit Blog
  • Follow MCottmeyer on Twitter
• Mike Kelly
  • Visit Blog
  • Follow Michael_D_Kelly on Twitter
• Mohammad Farooq
  • Follow agiletestware on Twitter
• Nana Yaa Mensah
  • Follow QAtweeter on Twitter
• Ocd Tester
  • Follow OcdTester on Twitter
• Panamo QA
  • Visit Blog
• Parimala Shankaraiah
  • Visit Blog
  • Follow CuriousTester on Twitter
• Mr. Penguin
  • Follow 0pensource on Twitter
• Phil Kirkham
  • Visit Blog
  • Follow PKirkham on Twitter
• Pradeep Soundararajan
  • Visit Blog
  • Follow TesterTested on Twitter
• Preston Redlon
  • Visit Blog
  • Follow PrestonKRedlon on Twitter
• QA Hates You
  • Visit Blog
  • Follow QAHatesYou on Twitter
• QA Tips
  • Visit Blog
  • Follow QATips on Twitter
• QA Witch
  • Follow QAWitch on Twitter
• QA World
  • Visit Blog
  • Follow QAWorld on Twitter
• Quality News
  • Visit Blog
  • Follow Quality_News on Twitter
• QualityPoint
  • Visit Blog
  • Follow QualityPoint on Twitter
• Replay Solutions
  • Follow ReplaySolutions on Twitter
• Richard Rostad
  • Visit Blog
  • Follow RichardRostad on Twitter
• Rob Lambert
  • Visit Blog
  • Follow Rob_Lambert on Twitter
• Rosie Sherry
  • Visit Blog
  • Follow RosieSherry on Twitter
• Roy Solomon
  • Follow RoySolomon on Twitter
• RubyTester
  • Visit Blog
  • Follow RubyTester on Twitter
• SCMagazine
  • Visit Site
  • Follow SCMagazine on Twitter
• Scott Barber
  • Visit Blog
  • Follow SBarber on Twitter
• Scott Price
  • Follow ScottPrice_Geek on Twitter
• Selena Delesie
  • Visit Blog
  • Follow SDelesie on Twitter
• Shrini Kulkarni
  • Visit Blog
  • Follow shrinik on Twitter
• Simon Godfrey
  • Visit Blog
  • Follow SGodfrey on Twitter
• Software Testing Stuff
  • Visit Blog
  • Follow SoftwareTestingStuff on Twitter
• SQAMatt
  • Follow SQAMatt on Twitter
• STAREAST
  • Visit Site
  • Follow STAREAST on Twitter
• STARWEST
  • Visit Site
  • Follow STARWEST on Twitter
• Stephan Schmidt
  • Visit Blog
  • Follow codemonkeyism on Twitter
• Steve Feloney
  • Visit Blog
• Sticky Minds
  • Visit Blog
  • Follow StickyMinds on Twitter
• STP Collaborative
  • Visit Blog
  • Follow STPCollab on Twitter
• Suhaim Abdussamad
  • Follow moorecats on Twitter
• Terry Morrish
  • Visit Blog
  • Follow thatqaguy on Twitter
• Test Plant
  • Visit Site
  • Follow TestPlant LTD on Twitter
• Test Republic
  • Visit Site
  • Follow TestRepublic on Twitter
• Testing Club
  • Follow TestingClub on Twitter
• Testing Concepts
  • Visit Blog
  • Follow TestingConcepts on Twitter
• Testing Geek
  • Visit Blog
  • Follow TestingGeek on Twitter
• Testing News
  • Visit Blog
  • Follow TestingNews on Twitter
• Testing Web Sites
  • Visit Blog
  • Follow TestingWebSites on Twitter
• Testomatix
  • Visit Site
  • Follow Testomatix on Twitter
• TestQs
  • Visit Site
  • Follow TestQs on Twitter
• Tim Koopmans
  • Visit Blog
  • Follow 90kts on Twitter
• Tim Coulter
  • Visit Blog
  • Follow TimothyJCoulter on Twitter
• Tobias Mayer
  • Visit Blog
  • Follow tobiasgmayer on Twitter
• Tomi Juhola
  • Visit Blog
  • Follow TomiJuhola on Twitter
• Tony Bruce
  • Visit Blog
  • Follow tonybruce77 on Twitter
• Trish Khoo
  • Visit Blog
• TweetMeme Technology
  • Visit Site
  • Follow TM_Technology on Twitter
• uTest
  • Visit Blog
  • Follow uTest on Twitter
• Venkat Reddy
  • Visit Blog
  • Follow VenkatReddyC on Twitter
• Ward Cunningham
  • Follow WardCunningham on Twitter
• WeekendTesters
  • Visit Site
  • Follow WeekendTesting on Twitter
• Yvette Francino
  • Visit Blog
  • Follow YvetteF on Twitter

Do you know of any other software testers/testing sites that should be added to the list?

Last Updated: 1/6/2010

Related posts:

1. 10+ Software Testing Blogs
2. 30+ Common Testing Mistakes
3. 100+ Open Source/Free Functional Testing Tools

I was cleaning out a folder on my machine when I stumbled on an oldie but goodie PDF called “Classic Testing Mistakes” by Brian Marick.

In the paper he states how he breaks the classic mistakes into 5 themes and how to resolve the mistakes:

1. The Role of Testing: who does the testing team serve, and how does it do that?
2. Planning the Testing Effort: how should the whole team’s work be organized?
3. Personnel Issues: who should test?
4. The Tester at Work: designing, writing, and maintaining individual tests.
5. Technology Rampant: quick technological fixes for hard problems.

Common mistakes that I see are:

• Communication: All team members need to be able to communicate with each other. A tester needs to know what to test, when to test, and the importance of what is to be tested
• Ability to change: Schedules & resources change so everyone on the team need to be able to understand that change is inevitable.
• Documentation: this is key especially in test steps, defect reports and findings reports. If you find a defect you have to be able to allow the next person to recreate it by providing them with detailed steps.
• Automation: Not everything can be automated nor should it be. Pick the tests that you find you are testing the most, smoke tests are great candidates for automation.
• Results: Don’t be fooled by the results. If a result shows that a test failed, investigate it before throwing up a flag.

All projects have a mistake of some kind so I thought I’d see what other people felt were common mistakes as well. I would like to thank all the contributors on Twitter & LinkedIn for taking the time to respond to the question/tweet. Below are the responses I received back with my comments (in purple of course):

Response from @michael_d_kelly on Twitter:

A common mistake I make is to get tunnel vision on the specific changes being made in a release, ignoring other quality criteria I agree, there are times that a person is so focused on one piece that they forget to look at the big picture.

Response from @aegeansys on Twitter:

Ignoring the fact that a certain bug fix is a feature, treating it as a normal bugfix and not adjusting the testing accordingly. When a bug/defect is a marked as a feature the requirements should also be updated. I have seen times where test cases are updated but the requirements are not.

Response from Vinodh Sen Ethirajulu on LinkedIn:

1. Functional test plan must be written by a tester and reviewed by a domain expert /business analyst. This must be withheld from the developer. This is intentional. A test plan should be sent to all project team members so everyone knows what the testing group is planning on testing.
2. Code checkin must be allowed only after all probablem scenarios is tested and passed in unit test. I agree but there will be times that an emergency fix needs to be put in. Also, I don’t think you can say no checkins allowed since there are pieces of code that are not tied directly to others. I think the response here should be that the code should not be deploed/deliverd to the test group until until testing has been completed
3. Some tools are there that can anticipate runtime errors better than a human reader/tester. All modules under current release must be subjected to such tools. code checkin must be allowed only after this. This would be more up to the company since not all tools are allowed within an organizations framework.
All three points are common sense. But some projects these are not followed strictly due to time constraint or resource constraint. Example of tool as in 3 is findbugs for eclipse. I am sure such tools exists for other platforms. Yes, there are tools out there but are not always allowed due to security.

Response from Geoff Feldman on LinkedIn:

1. Time and repetition is not relevant. Functions and boundary conditions on the functions including preserved state as it affects other functions is key. This occurs everywhere and all teams are impacted by this
2. Testers looking busy and unable to explain their coverage is a huge red flag. The bigger flag is when the defects keep showing up in production
3. Methodology is key. So is a testing process that begins with repeatable, maintained developer unit test. Agree, bigger issue I see is that not everyone agrees on which process/methodology is going to be followed. Everyone needs to agree and if it needs to be changed everyone should be made aware of the change.

Response from Rick Kiessig on LinkedIn:

1. Unit tests written by the same person that wrote the code, because they often use the same faulty logic in their tests as in the code itself. Solution: have some unit tests written by other developers or QA. Great solution
2. Not testing for quality: including performance, scalability and security I think that all applications should be tested for these items.
3. Focusing on cases that are really “self-testing,” rather than on the much-less-visible corner cases. Agree
4. Failing to include code coverage measurements and targets in unit test guidelines. Agree
5. Not following a coherent testing strategy. Agree, consistency is key.
6. Testing low-level components only, and forgetting to test the system as a whole. This is not something a test team should forget, if this happens I hope you bring it up right away to get it corrected.

Response from Bill Rinko-Gay on LinkedIn:

1. Not involving the QA group at the requirements phase. QA should be reviewing and commenting on requirements and design. Agree
2. Writing tests to the implementation rather than the requirements. This not only leads to trash as the implementation changes, but validates the wrong thing. Developers validate the implementation. Testers should validate the requirements. Testers should validate that the implementation worked by executing smoke tests and then running the test cases that are based on the requirements
3. Automating only after the software is stable. The bulk of passed test cases should be automated. Disagree, you cannot automate everything. Just because a test case passed doesn’t mean it should be automated. What should happen is when pieces of functionality are stable and being repeated in testing then they should be automated.
4. Automation experts should write code that can handle instability of early builds. Automation experts should have development expertise, not just record/playback. Use humans for inventing new tests, not running old ones. Partially agree. Yes, automation experts should have more than just record & playback expertise but if the code is not stable they cannot automate it. You automate the tests that you know what the expected result will be not ones that you think you know.

Response from Jonathan Ross on LinkedIn:

1. The tester should have an appreciation of the real world use cases of the deliverable being tested. Agree
2. If something feels wrong it is probably a bug of some nature. This is not always the case, there has been times that I thought something was wrong but when I went back to the requirements it was correct and not a bug/defect. If it feels wrong check the documentation before writing up the bug/defect.
3. Don’t wait until the end of the dev cycle to test. Small test cycles for small changes. Everyone on the project should agree to using an agile/iterative approach for this.
4. Not maintaining test documentation and scripts/ tools. This is huge especially since test teams are usually on multiple projects so up to date documentation is very important to keep track of what has been done, why, and what is left to be done.
You can make sure of this and other common errors with good planning, close work with R&D / Product and a highly motivated QA team who are kept passionate about their work.

Response from Jim Hanlon on LinkedIn:

1. The most common mistake in the testing process is underestimating the number of test cases required to adequately test the Application Under Test. This could be due to the requirements constantly changing and/or the test team is not involved in the requirements phase.
2. A corollary to the first is underestimating the time required to test the AUT.
3. No automated testing strategy: generation/execution/reporting. Agree
4. Some larger organizational issues: poor coordination with the requirements process; poor coordination with the debug/fixpack/build/release process. When this happens someone should step up and bring this up as an issue so it can be resolved

Response from Mike Sax on LinkedIn:

1. Not finding bugs doesn’t mean that they’re not there. It’s easy to start a testing effort seeing a small volume of defects and think “wow, it must be pretty clean!” Don’t wait until the mid point of the effort to get on top of the testing process to ensure that testers are effectively executing scripts and workflows, as well as evaluating the scripts and process to ensure an effective test cycle. Agree
2. Communication: Testers and developers who will be fixing bugs should be in close communication with each other. IM, phone, in person rather than email that can get lost in the shuffle. Too often, bugs fall into a “researching” status for days while emails shuffle back and forth to answer a question that could possibly have been addressed quickly in a face to face chat. Agree, should have touchpoint meetings as well to help resolve this
3. Effectively describing bugs: Especially for non QA professionals participating in testing. They need to be educated prior to beginning testing on the expected standards for creating defect records to ensure that as much information is relayed to the developers as possible (detailed error messages, screenshots, steps to reproduce, etc). Capturing all of this information initially cuts down on the need for back and forth communication between the developer and tester. Agree, all bugs should have detailed steps & screenshots so they can be recreated
4. Managing scope through triage: Effective defect triage will help to minimize the presence of enhancement requests (either intentional or inadvertent) that make it into a developer’s queue and impacts the timeline to complete defect remediation rather than flowing into the project’s change management process for new or changed functionality. Agree

Response from Walter Tyree on LinkedIn:

Testing on a hardware/software configuration that is not representative of what the users have. The computers that developers and testers have are quite often not what the user community will be running. Having a test lab (and forcing people to test in the lab) of machines that are configured for all of the scenarios within an Enterprise can save lots of time (Ghosting can really help here). Agree, testing should be completed on a variety of different configurations to make sure end users can use the application being delivered.

What common testing mistakes do you see?  How do you resolve theses issues so they don’t keep happening?  What are your thoughts on the responses that are noted in this blog?  Do you agree/disagree?

Related posts:

1. 15+ Open Source Test Management Tools
2. 100+ Open Source/Free Functional Testing Tools
3. 100+ Open Source/Free Security Tools

Mashable’s 2009 Open Web Awards is still open for nominations. Nominations started on October 14, 2009 and last until November 15, 2009. Voting is from November 18th – December 13th, 2009. Winner Announcements will be on Tuesday, December 15th, 2009.

You can nominate people/companies for various different categories ranging from:

• Brand/Startup/Company/Agency
• FaceBook Specific
• Media
• Mobile
• People
• Photos and Videos
• Twitter Specific

There are a total of 50 categories, Mashable encourages you to not only nominate across all categories, but to nominate once a day per category. Nominations will remain open until 11:59 PM, ET on Sunday, November 15th.

I wonder how many nominations I would get for “Most Interesting Twitter User to Follow” & “Twitter User of the Year“

Related posts:

1. Twellow can be used to increase connections
2. 28 sites to help you search Twitter and more
3. 100+ Software Testing Blogs and Tweeters

Security testing is a process to determine that an IS (Information System) protects data and maintains functionality as intended.

The six concepts that need to be covered by security testing are: confidentiality, integrity, authentication, authorization, availability, and non-repudiation.

Confidentiality: A security measure which protects against the disclosure of information to parties other than the intended recipient(s). Often ensured by means of encoding, using a defined algorithm and some secret information known only to the originator of the information and the intended recipient(s) (a process known as cryptography) but that is by no means the only way of ensuring confidentiality.

Integrity: A measure intended to allow the receiver to determine that the information which it receives has not been altered in transit or by other than the originator of the information. Integrity schemes often use some of the same underlying technologies as confidentiality schemes, but they usually involve adding additional information to a communication to form the basis of an algorithmic check rather than encoding all of the communication.

Authentication: A measure designed to establish the validity of a transmission, message, or originator. It allows a receiver to have confidence that the information it receives originated from a specific known source.

Authorization: The process of determining that a requester is allowed to receive a service or perform an operation.

Availability: Assuring information and communications services will be ready for use when expected. Information must be kept available to authorized persons when they need it.

Non-repudiation: A measure intended to prevent the later denial that an action happened, or a communication took place, etc. In communication terms, this often involves the interchange of authentication information combined with some form of provable time stamp.

Below are some open source/free tools that can help you with security testing as well as tools that will keep your system secure. Please use these tools ONLY for good.

• Aircrack: A suite of tools for 802.11a/b/g WEP and WPA cracking. It can recover a 40 through 512-bit WEP key once enough encrypted packets have been gathered. It can also attack WPA 1 or 2 networks using advanced cryptographic methods or by brute force. The suite includes airodump (an 802.11 packet capture program), aireplay (an 802.11 packet injection program), aircrack (static WEP and WPA-PSK cracking), and airdecap (decrypts WEP/WPA capture files).
• AirSnort: A wireless LAN (WLAN) tool that recovers encryption keys. Operates by passively monitoring transmissions, computing the encryption key when enough packets have been gathered
• Angry IP Scanner: A small open source Java application which performs host discovery (”ping scan”) and port scans. The old 2.x release was Windows-only, but the new 3.X series runs on Linux, Mac, or Windows as long as Java is installed. Version 3.X omits the vampire zebra logo. As with all connect()-based scanners, performance on Windows XP SP2 and Vista can be poor due to limitations added to tcpip.sys. The Angry FAQ provides details and workarounds. A short review was posted to nmap-dev
• Audit Record Generation and Utilization System (Argus): A fixed-model Real Time Flow Monitor designed to track and report on the status and performance of all network transactions seen in a data network traffic stream. Argus provides a common data format for reporting flow metrics such as connectivity, capacity, demand, loss, delay, and jitter on a per transaction basis. The record format that Argus uses is flexible and extensible, supporting generic flow identifiers and metrics, as well as application/protocol specific information
• Babel Enterprise: Manages the risk, dividing it by domains (groups or organizations), assets and policies. With all this, it can be checked, point by point the fully compliance of a security regulation, such as UNE-ISO/IEC 27001 or other ones that depend on this such as LOPD, SOX, etc. Requirement: Linux, Solaris, WinXP, HP-UX, IBM AIX
• Basic Analysis and Security Engine (BASE): A PHP-based analysis engine to search and process a database of security events generated by various IDSs, firewalls, and network monitoring tools. Its features include a query-builder and search interface for finding alerts matching different patterns, a packet viewer/decoder, and charts and statistics based on time, sensor, signature, protocol, IP address, etc.
• Bastille: Program “locks down” an operating system, proactively configuring the system for increased security and decreasing its susceptibility to compromise. Bastille can also assess a system’s current state of hardening, granularity reporting on each of the security settings with which it works. Bastille currently supports the Red Hat (Fedora Core, Enterprise, and Numbered/Classic), SUSE, Debian, Gentoo, and Mandrake distributions, along with HP-UX and Mac OS X. Bastille’s focuses on letting the system’s user/administrator choose exactly how to harden the operating system. In its default hardening mode, it interactively asks the user questions, explains the topics of those questions, and builds a policy based on the user’s answers. It then applies the policy to the system. In its assessment mode, it builds a report intended to teach the user about available security settings as well as inform the user as to which settings have been tightened
• Brute Force Binary Tester (BFBTester): Good for doing quick, proactive security checks of binary programs. BFBTester will perform checks of single and multiple argument command line overflows and environment variable overflows. It can also watch for tempfile creation activity to alert the user of any programs using unsafe tempfile names.Requirement: POSIX, BSD, FreeBSD, OpenBSD, Linux
• Burp Suite: An integrated platform for attacking web applications. It contains all of the Burp tools with numerous interfaces between them designed to facilitate and speed up the process of attacking an application. All tools share the same robust framework for handling HTTP requests, persistence, authentication, downstream proxies, logging, alerting and extensibility. Allows you to combine manual and automated techniques to enumerate, analyze, scan, attack and exploit web applications. The various Burp tools work together effectively to share information and allow findings identified within one tool to form the basis of an attack using another
• Cain & Abel: This Windows-only password recovery tool handles an enormous variety of tasks. It can recover passwords by sniffing the network, cracking encrypted passwords using Dictionary, Brute-Force and Cryptanalysis attacks, recording VoIP conversations, decoding scrambled passwords, revealing password boxes, uncovering cached passwords and analyzing routing protocols.
• Cheops-ng: A Network management tool for mapping and monitoring your network. It has host/network discovery functionality as well as OS detection of hosts. Cheops-ng has the ability to probe hosts to see what services they are running. On some services, cheops-ng is actually able to see what program is running for a service and the version number of that program
• chkrootkit: A flexible, portable tool that can check for many signs of rootkit intrusion on Unix-based systems. Its features include detecting binary modification, utmp/wtmp/lastlog modifications, promiscuous interfaces, and malicious kernel modules. Requirements: Linux, Mac OS X
• Clam AntiVirus (ClamAV): An open source (GPL) anti-virus toolkit for UNIX, designed especially for e-mail scanning on mail gateways. It provides a number of utilities including a flexible and scalable multi-threaded daemon, a command line scanner and advanced tool for automatic database updates
• Codenomicon Robust Open Source Software (CROSS): Program is designed to help open source projects fix critical flaws in their code. Codenomicon’s CROSS program provides open source projects with full access to its award-winning DEFENSICS testing solutions, helping the projects find and fix a large number of critical flaws very rapidly. Requirement: 130 protocol interfaces and formats

• Dsniff: A suite of powerful network auditing and penetration-testing tools. Includes many tools. dsniff, filesnarf, mailsnarf, msgsnarf, urlsnarf, and webspy passively monitor a network for interesting data (passwords, e-mail, files, etc.). arpspoof, dnsspoof, and macof facilitate the interception of network traffic normally unavailable to an attacker (e.g, due to layer-2 switching). sshmitm and webmitm implement active monkey-in-the-middle attacks against redirected ssh and https sessions by exploiting weak bindings in ad-hoc PKI. It handles pretty much all of your password sniffing needs.
• EtherApe: A graphical network monitor for Unix modeled after etherman. Features link layer, IP and TCP modes, EtherApe displays network activity graphically with a color coded protocols display. Hosts and links change in size with traffic. It supports Ethernet, FDDI, Token Ring, ISDN, PPP and SLIP devices. It can filter traffic to be shown, and can read traffic from a file as well as live from the network
• Ettercap: A terminal-based network sniffer/interceptor/logger for ethernet LANs. It supports active and passive dissection of many protocols (even ciphered ones, like ssh and https). Data injection in an established connection and filtering on the fly is also possible, keeping the connection synchronized. Many sniffing modes were implemented to give you a powerful and complete sniffing suite. Plugins are supported. It has the ability to check whether you are in a switched LAN or not, and to use OS fingerprints (active or passive) to let you know the geometry of the LAN. Requirement: Windows/Linux/Mac OS X
• Flawfinder: Program that scans C/C++ source code and reports potential security flaws. By default, it sorts its reports by risk level (the riskiest operations in the code are listed first). Requirement: Python 1.5 or greater
• fping: A ping(1) like program which uses the Internet Control Message Protocol (ICMP) echo request to determine if a host is up. fping is different from ping in that you can specify any number of hosts on the command line, or specify a file containing the lists of hosts to ping. Instead of trying one host until it timeouts or replies, fping will send out a ping packet and move on to the next host in a round-robin fashion. If a host replies, it is noted and removed from the list of hosts to check. If a host does not respond within a certain time limit and/or retry limit it will be considered unreachable.
• fragroute: Features a simple ruleset language to delay, duplicate, drop, fragment, overlap, print, reorder, segment, source-route, or otherwise monkey with all outbound packets destined for a target host, with minimal support for randomized or probabilistic behaviour.
• Fragrouter: A one-way fragmenting router – IP packets get sent from the attacker to the Fragrouter, which transforms them into a fragmented data stream to forward to the victim
• Gendarme: An extensible rule-based tool to find problems in .NET applications and libraries. Gendarme inspects programs and libraries that contain code in ECMA CIL format (Mono and .NET) and looks for common problems with the code, problems that compilers do not typically check or have not historically checked. Requirement: .NET (Mono or MS runtime)
• GnuPG: GnuPG allows to encrypt and sign your data and communication, features a versatile key managment system as well as access modules for all kind of public key directories. GnuPG, also known as GPG, is a command line tool with features for easy integration with other applications.
• Helix: A customized distribution of the Knoppix Live Linux CD. Helix is more than just a bootable live CD. You can still boot into a customized Linux environment that includes customized Linux kernels, excellent hardware detection and many applications dedicated to Incident Response and Forensics. Helix has been designed very carefully to NOT touch the host computer in any way and it is forensically sound. Helix will not auto mount swap space, or auto mount any attached devices. Helix also has a special Windows autorun side for Incident Response and Forensics
• Honeyd: A small daemon that creates virtual hosts on a network. The hosts can be configured to run arbitrary services, and their TCP personality can be adapted so that they appear to be running certain versions of operating systems. Honeyd enables a single host to claim multiple addresses on a LAN for network simulation. It is possible to ping the virtual machines, or to traceroute them. Any type of service on the virtual machine can be simulated according to a simple configuration file. It is also possible to proxy services to another machine rather than simulating them
• Hping2: Handy little utility assembles and sends custom ICMP, UDP, or TCP packets and then displays any replies. It was inspired by the ping command, but offers far more control over the probes sent. It also has a handy traceroute mode and supports IP fragmentation. This tool is particularly useful when trying to traceroute/ping/probe hosts behind a firewall that blocks attempts using the standard utilities. This often allows you to map out firewall rulesets. It is also great for learning more about TCP/IP and experimenting with IP protocols. Requirement: Windows/Linux/Mac OS X
• ike-scan: Exploits transport characteristics in the Internet Key Exchange (IKE) service, the mechanism used by VPNs to establish a connection between a server and a remote client. It scans IP addresses for VPN servers by sending a specially crafted IKE packet to each host within a network. Most hosts running IKE will respond, identifying their presence. The tool then remains silent and monitors retransmission packets. These retransmission responses are recorded, displayed and matched against a known set of VPN product fingerprints. Ike-scan can VPNs from manufacturers including Checkpoint, Cisco, Microsoft, Nortel, and Watchguard
• IP Filter: Portable UNIX Packet Filter. Software package that can be used to provide network address translation (NAT) or firewall services. It can either be used as a loadable kernel module or incorporated into your UNIX kernel; use as a loadable kernel module where possible is highly recommended. Scripts are provided to install and patch system files, as required. IP Filter is distributed with FreeBSD, NetBSD, and Solaris.
• John the Ripper: A fast password cracker, currently available for many flavors of Unix (11 are officially supported, not counting different architectures), DOS, Win32, BeOS, and OpenVMS. Its primary purpose is to detect weak Unix passwords. It supports several crypt(3) password hash types which are most commonly found on various Unix flavors, as well as Kerberos AFS and Windows NT/2000/XP LM hashes
• Kismet: A console (ncurses) based 802.11 layer2 wireless network detector, sniffer, and intrusion detection system. It identifies networks by passively sniffing (as opposed to more active tools such as NetStumbler), and can even decloak hidden (non-beaconing) networks if they are in use. It can automatically detect network IP blocks by sniffing TCP, UDP, ARP, and DHCP packets, log traffic in Wireshark/TCPDump compatible format, and even plot detected networks and estimated ranges on downloaded maps. Requirements: Windows/Linux/Mac OS X
• Knoppix: Consists of a representative collection of GNU/Linux software, automatic hardware detection, and support for many graphics cards, sound cards, SCSI and USB devices and other peripherals. KNOPPIX can be used as a productive Linux system for the desktop, educational CD, rescue system, or as many nmap survey takers attest, a portable security tool.
• Libwhisker: A Perl module geared geared towards HTTP testing. It provides functions for testing HTTP servers for many known security holes, particularly the presence of dangerous CGIs
• Metasploit: The framework is an advanced open-source platform for developing, testing, and using exploit code. This project initially started off as a portable network game and has evolved into a powerful tool for penetration testing, exploit development, and vulnerability research. Requirement: Win32 / UNIX
• Microsoft Baseline Security Analyzer (MBSA): Tool designed for the IT professional that helps small- and medium-sized businesses determine their security state in accordance with Microsoft security recommendations and offers specific remediation guidance. Improve your security management process by using MBSA to detect common security misconfigurations and missing security updates on your computer systems.
• Nagios: An open source host, service and network monitoring program. It watches hosts and services that you specify, alerting you when things go bad and when they get better. Some of its many features include monitoring of network services (smtp, pop3, http, nntp, ping, etc.), monitoring of host resources (processor load, disk usage, etc.), and contact notifications when service or host problems occur and get resolved (via email, pager, or user-defined method)
• NBTscan: A program for scanning IP networks for NetBIOS name information. It sends NetBIOS status query to each address in supplied range and lists received information in human readable form. For each responded host it lists IP address, NetBIOS computer name, logged-in user name and MAC address.
• Netfilter: A powerful packet filter implemented in the standard Linux kernel. The userspace iptables tool is used for configuration. It now supports packet filtering (stateless or stateful), all kinds of network address and port translation (NAT/NAPT), and multiple API layers for 3rd party extensions. It includes many different modules for handling unruly protocols such as FTP
• NetStumbler: A Free Windows 802.11 Sniffer. Known for finding open wireless access points (”wardriving”). They also distribute a WinCE version for PDAs and such named Ministumbler.
• Nikto: Web server scanner which performs comprehensive tests against web servers for multiple items, including over 3200 potentially dangerous files/CGIs, versions on over 625 servers, and version specific problems on over 230 servers.Requirement: Windows/UNIX
• Ntop: Shows network usage in a way similar to what top does for processes. In interactive mode, it displays the network status on the user’s terminal. In Web mode, it acts as a Web server, creating an HTML dump of the network status. It sports a NetFlow/sFlow emitter/collector, an HTTP-based client interface for creating ntop-centric monitoring applications, and RRD for persistently storing traffic statistics.

• Oedipus: A web application security analysis and testing suite written in Ruby. It is capable of parsing different types of log files off-line and identifying security vulnerabilities. Using the analyzed information, Oedipus can dynamically test web sites for application and web server vulnerabilities. Requirement: OS Independent
• OllyDbg: A 32-bit assembler level analyzing debugger for Microsoft Windows. Emphasis on binary code analysis makes it particularly useful in cases where source is unavailable. OllyDbg features an intuitive user interface, advanced code analysis capable of recognizing procedures, loops, API calls, switches, tables, constants and strings, an ability to attach to a running program, and good multi-thread support. OllyDbg is free to download and use but no source code is provided
• OpenBSD: Produces a free, multi-platform 4.4BSD-based UNIX-like operating system. Our efforts emphasize portability, standardization, correctness, proactive security and integrated cryptography. OpenBSD supports binary emulation of most programs from SVR4 (Solaris), FreeBSD, Linux, BSD/OS, SunOS and HP-UX
• OpenBSD Packet Filter (OpenBSD PF): Handles network address translation, normalizing TCP/IP traffic, providing bandwidth control, and packet prioritization. It also offers some eccentric features, such as passive OS detection.
• OpenSSH: A FREE version of the SSH connectivity tools that technical users of the Internet rely on. Users of telnet, rlogin, and ftp may not realize that their password is transmitted across the Internet unencrypted, but it is. OpenSSH encrypts all traffic (including passwords) to effectively eliminate eavesdropping, connection hijacking, and other attacks. Additionally, OpenSSH provides secure tunneling capabilities and several authentication methods, and supports all SSH protocol versions. The OpenSSH suite replaces rlogin and telnet with the ssh program, rcp with scp, and ftp with sftp. Also included is sshd (the server side of the package), and the other utilities like ssh-add, ssh-agent, ssh-keysign, ssh-keyscan, ssh-keygen and sftp-server
• Open Source Security Testing Methodology Manual (OSSTMM): This manual is to set forth a standard for Internet security testing.
• OSSEC HIDS: An Open Source Host-based Intrusion Detection System.
  OSSEC HIDS performs log analysis, integrity checking, rootkit detection, time-based alerting and active response. In addition to its IDS functionality, it is commonly used as a SEM/SIM solution. Because of its powerful log analysis engine, ISPs, universities and data centers are running OSSEC HIDS to monitor and analyze their firewalls, IDSs, web servers and authentication logs
• Paros: This is for people who need to evaluate the security of their web applications. It is completely written in Java. All HTTP and HTTPS data between server and client, including cookies and form fields, can be intercepted and modified. Requirement: Cross-platform, Java JRE/JDK 1.4.2 or above
• P0f: A versatile passive OS fingerprinting tool. Able to identify the operating system of a target host simply by examining captured packets even when the device in question is behind an overzealous packet firewall. P0f does not generate ANY additional network traffic, direct or indirect. No name lookups, no mysterious probes, no ARIN queries, nothing. In the hands of advanced users, P0f can detect firewall presence, NAT use, existence of load balancers, and more!
• PuTTY: A free implementation of Telnet and SSH for Win32 and Unix platforms, along with an xterm terminal emulator. Use of PuTTY, PSCP, PSFTP and Plink is illegal in countries where encryption is outlawed.
• Root Kit Hunter: A Unix scanning tool that checks for signs of various pieces of nasty software on your system like rootkits, backdoors and local exploits. It runs many tests, including MD5 hash comparisons, default filenames used by rootkits, wrong file permissions for binaries, and suspicious strings in LKM and KLD modules
• Sam Spade: Provides a consistent GUI and implementation for many handy network query tasks. It was designed with tracking down spammers in mind, but can be useful for many other network exploration, administration, and security tasks. It includes tools such as ping, nslookup, whois, dig, traceroute, finger, raw HTTP web browser, DNS zone transfer, SMTP relay check, website search, and more. Non-Windows users can enjoy online versions of many of their tools
• Scapy: A powerful interactive packet manipulation tool, packet generator, network scanner, network discovery tool, and packet sniffer. It provides classes to interactively create packets or sets of packets, manipulate them, send them over the wire, sniff other packets from the wire, match answers and replies, and more. Interaction is provided by the Python interpreter, so Python programming structures can be used (such as variables, loops, and functions). Report modules are possible and easy to make.
• Socat: A utility similar to the venerable Netcat that works over a number of protocols and through a files, pipes, devices (terminal or modem, etc.), sockets (Unix, IP4, IP6 – raw, UDP, TCP), a client for SOCKS4, proxy CONNECT, or SSL, etc. It provides forking, logging, and dumping, different modes for interprocess communication, and many more options. It can be used, for example, as a TCP relay (one-shot or daemon), as a daemon-based socksifier, as a shell interface to Unix sockets, as an IP6 relay, for redirecting TCP-oriented programs to a serial line, or to establish a relatively secure environment (su and chroot) for running client or server shell scripts with network connections
• SPIKE Proxy: An open source HTTP proxy for finding security flaws in web sites. It is part of the Spike Application Testing Suite and supports automated SQL injection detection, web site crawling, login form brute forcing, overflow detection, and directory traversal detection Requirement: Python and pyOpenSSL
• Sguil: (pronounced sgweel) Built by network security analysts for network security analysts. Sguil’s main component is an intuitive GUI that provides realtime events from Snort/barnyard. It also includes other components which facilitate the practice of Network Security Monitoring and event driven analysis of IDS alerts
• Stunnel: Designed to work as an SSL encryption wrapper between remote client and local (inetd-startable) or remote server. It can be used to add SSL functionality to commonly used inetd daemons like POP2, POP3, and IMAP servers without any changes in the programs’ code. It will negotiate an SSL connection using the OpenSSL or SSLeay libraries
• SuperScan: A free Windows-only closed-source TCP/UDP port scanner by Foundstone. It includes a variety of additional networking tools such as ping, traceroute, http head, and whois
• Sysinternals: Provides many small windows utilities that are quite useful for low-level windows hacking. Some are free of cost and/or include source code, while others are proprietary.
• Tcpdump: This is the IP sniffer we all used before Ethereal (Wireshark) came on the scene, and many of us continue to use it frequently. It may not have the bells and whistles (such as a pretty GUI or parsing logic for hundreds of application protocols) that Wireshark has, but it does the job well and with fewer security holes. It also requires fewer system resources. While it doesn’t receive new features often, it is actively maintained to fix bugs and portability problems. It is great for tracking down network problems or monitoring activity. Requirement: Windows/Linux/ Mac OS X
• THC Hydra: A Fast network authentication cracker which supports many different services. It can perform rapid dictionary attacks against more then 30 protocols, including telnet, ftp, http, https, smb, several databases, and much more.
• Tor: A toolset for a wide range of organizations and people that want to improve their safety and security on the Internet. Using Tor can help you anonymize web browsing and publishing, instant messaging, irc, ssh, and other applications that use the TCP protocol. Tor also provides a platform on which software developers can build new applications with built-in anonymity, safety, and privacy features.
• TrueCrypt: An open source disk encryption system. Users can encrypt entire filesystems, which are then on-the-fly encrypted/decrypted as needed without user intervention beyond entering their passphrase intially. A clever hidden volume feature allows you to hide a 2nd layer of particularly sensitive content with plausible deniability about whether it exists. Then if you are forced to give up your passphrase, you give them the first-level secret. Even with that, attackers cannot prove that a second level key even exists.
• Unicornscan: A User-land Distributed TCP/IP stack for information gathering and correlation. It is intended to provide a researcher a superior interface for introducing a stimulus into and measuring a response from a TCP/IP enabled device or network. Some of its features include asynchronous stateless TCP scanning with all variations of TCP flags, asynchronous stateless TCP banner grabbing, and active/passive remote OS, application, and component identification by analyzing responses
• WebScarab: A loose suite of web application security assessment tools written entirely in Java. It is a tool primarily designed to be used by developers who can write code themselves. Requirement: OS Indpendent
• Wikto: Tool that checks for flaws in webservers. It provides much the same functionality as Nikto but adds various interesting pieces of functionality, such as a Back-End miner and close Google integration. Wikto is written for the MS .NET environment and registration is required to download the binary and/or source code.
• Wireshark: Wireshark, formerly known as Ethereal, is used by network professionals around the world for troubleshooting, analysis, software and protocol development, and education. It has all of the standard features you would expect in a protocol analyzer, and several features not seen in any other product. Requirement: Unix, Linux, and Windows
• Yersinia: A multi-protocol low-level attack tool useful for penetration testing. It is capable of many diverse attacks over multiple protocols, such as becoming the root role in the Spanning Tree (Spanning Tree Protocol), creating virtual CDP (Cisco Discovery Protocol) neighbors, becoming the active router in a HSRP (Hot Standby Router Protocol) scenario, faking DHCP replies, and other low-level attacks

Note: Most of the product descriptions were taken from the applications site. Applications were listed in alphabetical order so there are no favorites since the tool you pick is based on need.

What are your thoughts on these tools?  Do you know of any others that I may have missed?

Related posts:

1. 15+ Open Source Test Management Tools
2. 100+ Open Source/Free Functional Testing Tools
3. 50+ Open Source Tracking Database Tools

Test Management tools are very important to any test team. Test teams use these tools to help capture requirements, design test cases, map test cases to requirements, test execution reports and much more. Companies may use one to many tools for this, which range from very expensive to open source. My advice would be to pick a tool that can meet most of your current and near future needs.

Below is a list of some open source/free tools that may come in handy.

• Bugzilla Testopia: A test case management extension for Bugzilla. It is designed to be a generic tool for tracking test cases, allowing for testing organizations to integrate bug reporting with their test case run results. Though it is designed with software testing in mind, it can be used to track testing on virtually anything in the engineering process. Requirement: Bugzilla
• Data Generator: The Data Generator is a free, open source script written in JavaScript, PHP and MySQL that lets you quickly generate large volumes of custom data in a variety of formats for use in testing software, populating databases and more.
• Fitnesse: A collaborative testing and documentation tool. It provides a very simple way for teams to collaboratively create documents, specify tests, and run those tests. Requirement: Mac, Windows, POSIX
• Kineo Open Source: Advises on the best open source tools tailored for a specific group of users
• Mantis: Can be used for bug tracking as well.
• MTS: Multi-TeSTer: A simple domain-specific language for maintaining tests that supports two desirable properties: – single source: the command, input, and expected output are all in one file. – controlled experimentation: multiple slightly different tests can be generated from the same source with different expected outputs. Requirement: Unix command line tool; written in Perl
• qaManager: A web based application used for keeping track of engineering and customer releases of Software Projects, Resource allocations and related information. It helps QA Managers to keep track of releases
  effectively.Powered by OpenXava and Java.
• QATraq: Covers everything from defining test plans to writing test cases and recording results. Requirement: Windows, BSD, Linux, SunOS/Solaris
• Radi: Radi-testdir is a lightweight test director. Radi supports test director features like configuring the test plan, updating (create/edit) the test results for the test image/build stores in the image results set. Requirement: All POSIX (Linux/BSD/UNIX-like OSes)
• rth: web-based tool designed to manage requirements, tests, test results, and defects throughout the application life cycle. The tool provides a structured approach to software testing and increases the visibility of the testing process by creating a common repository for all test assets including requirements, test cases, test plans, and test results. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP), All POSIX (Linux/BSD/UNIX-like OSes), IBM AIX
• 
• RTH-Turbo: Optimized version of RTH, a testing management tool, that allows: requirement management; test case management; defect tracking; create test plans; analyze your test results. This project was originally created from RTH version 1.2, and this new branch and version is more powerful and optimized. Requirement: PHP, MySql and Apache
• Salome-TMF: An independent Test Management Tool, which helps you to manage your entire testing process by creating tests, executing manual or automatic tests, tracking results, managing requirements and defects and producing HTML documentation. Salome-TMF is compatible with Junit, Abbot and Beanshell to define your automatic tests, and with Bugzilla and Mantis to manage your defects. Salome-TMF can also be extended by plug-in according to your requirements. Requirement: Java
• TCW: Test Case Web (TCW) is an online TCM system built with PHP and a SQL backend. It provides an efficient means for generation, organization, and execution reporting of test cases among projects and by multiple testers and versions. It provides various at-a-glance views of the test suite for easy status determination and test suite navigation. TCW also provides basic reporting capabilities and per-project access control. Requirement:Any (PHP/SQL/Apache)
• Tesly: Tesly is a Web application written in PHP that helps you create, execute, and report on test plans. QA leaders can track the progress of testing as testers use the interface to report completion of test cases. Requirement: OS Independent
• Test Environment Toolkit: Open source version of TETware, a proprietary multi-platform test framework for test suite management (administration, reporting and sequencing of tests). This open-source version is for Linux and UNIX system users only. The differences between the open source version and the other version for which a commercial support offering is available is in platform support (WIN32 and Java) rather than functionality. Requirement: Linux or Unix
• Testitool: A Web-based application for QA test planning. It creates a test plan and populates it with test cases, maps test cases to functional requirements, instantiates a test plan, begins executing test cases and marks them as successful or failed, generates reports on your test plans, copies test plans and test cases, and tailors test plan instances by adding and removing test cases from them.Requirement: Apache, PHP, MySQL
• TestLink: Web-based test management and test execution system allowing QA teams to create, manage, execute and track test cases and organize them into test plans. Requirement: Apache, MySQL, PHP
• TestMaster: A testcase management, logging, reporting and test automation tool, similar to the commercial product Test Director. Features: Progress stats, reports, test case import from CSV,doc,web or SQL, STAF plugin. Requirement: Linux, Apache, PostgreSQL
• WebTst: Tool aimed at creating and managing user-centric testing
• XQual Studio (XStudio):  A free 100% graphical and modular in design test management application that handles the complete life-cycle of your QA/testing projects from end to end: users, requirements, specifications, development projects (scrum oriented), SUTs, tests, tesplans, test reports and test campaigns

Please comment below on your thoughts/experience on any of the tools listed above and/or if you know of any other tools that should be added.

Related posts:

1. 100+ Open Source/Free Functional Testing Tools
2. 100+ Open Source/Free Security Tools
3. 10+ Open Source Link Checking Tools

It is very important to make sure that your application functions as expected. There may be times that you add one little piece of code and all of a sudden other parts of the application no longer works. You may not have time/capacity to manually go back and regression test all the pieces of your application to make sure they are up to par. Companies use a variety of different testing tools for regression testing. There are lots of tools out there from very expensive to open source.

Below is a list of some open source/free tools that may come in handy.

Please keep in mind that every application is different so the tool you pick from one application may not be the same tool that you pick for another. My advice would be to pick a tool that can meet most of your current and near future needs.

• Abbot Java GUI Test Framework: The Abbot framework provides automated event generation and validation of Java GUI components, improving upon the very rudimentary functions provided by the java.awt.Robot class (A Better ‘Bot). The framework may be invoked directly from Java code or accessed without programming through the use of scripts. It is suitable for use both by developers for unit tests and QA for functional testing.
• actiWate: Java-based Web application testing environment from Actimind Inc. Advanced framework for writing test scripts in Java (similar to open-source frameworks like HttpUnit, HtmlUnit etc. but with extended API), and Test Writing Assistant – Web browser plug-in module to assist the test writing process. Freeware.
• Anteater: A testing framework designed around Ant, from the Apache Jakarta Project. It provides an easy way to write tests for checking the functionality of a Web application or of an XML Web service.
• Apodora: A framework/tool for automating functional testing of web applications. It provides the user with programmatic control of the web browser allowing them to interact directly with the browser’s user interface. It uses a database backend in order to remember how to find your html elements. This also makes your scripts easier to maintain. Requirement: Windows, IE6
• Arbiter: Document based acceptance tester. Similar to FIT in goal. Manages requirements documents in Word or RTF format that are created jointly by customer and developer. Requirements are parsed to extract a glossary and test suite.
• Autonet:Autonet is a GUI network
  test platform, internally it’s based on CLI to communicate with devices. It can help you to arrange test cases, setup commands to devices,run commands to check results and record test results. Requirement: windows, linux and any other platform which support tcl
• AutoTestFlash: Allows the recording and playback of tests written in Flash and Flex. The tool website provides a live sample. Requirement: Windows / Flash
• Avignon: An acceptance test system that allows you to write executable tests in a language that you define. It uses XML to define the syntax of the language but, if you choose to extend the language, leaves the semantics of the tests up to you. Avignon includes modules for testing HTML applications (through either IE or FireFox), Swing and .NET WinForm applications. Requirement: Java (MS Windows only for .NET testing)
• Blerby Test Runner: Ajax test runner for php. Currently supports simpletest and phpunit 3.x. Allows developers to refactor code while being able to receive instant feedback on their changes. Tracks test dependencies and automatically re-runs appropriate tests upon source changes. Requirement: Windows, *nix, apache
• Canoo WebTest: Used for functional testing of web pages, WebTest is an open source testing framework built on top of HttpUnit. It allows tests to be defined in XML as Ant targets. Requirement: JDK 1.2 and ANT v1.3
• Celerity: a JRuby wrapper around HtmlUnit – a headless Java browser with JavaScript support. It provides a simple API for programmatic navigation through web applications. Celerity aims at being API compatible with Watir.
• Concordion: Framework for Java that lets you turn a plain English description of a requirement into an automated test. Concordion specifications are active. Behind the scenes, they are linked to the system under test and therefore do not go out-of-date. If a change is made to the system’s behaviour then the tests associated with the relevant specification will fail and let you know.Requirement: Java 1.5 or above
• Crosscheck: Framework for verifying your in-browser javascript. It helps you ensure that your code will run in many different browsers such as Internet Explorer and Firefox, but without needing installations of those browsers. The only thing you need is a Java Virtual Machine.Requirement: Java Virtual Machine
• csvdiff: A Perl script for comparing two files of comma-separated values with each other. In contrast to standard diff, it will show the number of the record where the difference occurs, the column number, and (if provided) the fieldname which is different. The separator can be set to any value you want, not just a comma. It also provides support for multiple column keys, the ability to ignore case and trim leading/tailing spaces, and the ability to ignore selected columns such as timestamps. Requirement: Everywhere you can install perl
• CubicTest: A graphical Eclipse plug-in for writing Selenium and Watir tests. It makes web tests faster and easier to write, and provides abstractions to make tests more robust and reusable. CubicTest’s test editor is centered around pages/states and transitions between these pages/states. The model is intuitive for both Ajax and traditional web applications and supports most user interaction types. CubicTest features an innovative test recorder and test runner based on Selenium RC which are fully integrated with the graphical test editor. Tests can also run standalone from Maven 2. Requirement: Eclipse plug-in
• DBFeeder: With DBFeeder you can automatically generate testdata for Oracle Databases which fits primary and foreign keys of tables. A file-based configuration system allows in-depth customization of the generated data.
• DbFit: Extension to FIT/FitNesse for test-driven database development. Enables developers to manipulate database objects in a relational/ tabular form, making database testing and management much easier then with xUnit-style tools. Requirement: Java/.NET
• DejaGnu: Framework for testing applications such as Tcl, C, C++, Java and network applications and cross testing of embedded systems. Its purpose is to provide a single front end for all tests. Think of it as a custom library of Tcl procedures crafted to support writing a test harness. Requirement: MacOS, Windows, POSIX
• Dogtail: A GUI test tool and automation framework written in Python. It uses accessibility technologies to communicate with desktop applications. Dogtail scripts are written in Python and executed like any other Python program. Requirement: Python 2.3 or higher
• Doit: Simple Web Application Testing: Scripting tool and language for testing web applications that use forms. Doit can generate random or sequenced form fill-in information, report results (into a database, file, or
  stdout), filter HTML results, and compare results to previous results, without having to manually use a web browser. It uses a console-based web client tool (like Curl or Wget) to send and receive HTTP requests and responses respectively. Requirement: You must have Perl 5 or greater and the appropriate Perl modules (detailed in Doit manual) installed on your system before you can use SPL.
• Eclipse TPTP: The Eclipse test and performance tools platform (TPTP) provides support for three types of testing: Performance testing of HTTP applications, JUnit testing and manual testing. Although each of these areas of testing has its own unique set of tasks and concepts, two sets of topics are common to all three types: creation and use of datapools, and creation of test deployments. Requirement: Eclipse
• EMOS Framework: A simple yet powerful environment for development of automated WinRunner? tests. Like most frameworks of this sort EMOS Framework separates test data from the test code in order to simplify and speed up test development, increase robustness of the produced solution, and empower non-programmers towards test automation. It is almost completely written in WinRunner’s own scripting language, TSL.Requirement: Mercury WinRunner, All 32-bit MS Windows (95/98/NT/2000/XP)
• Enterprise Web Test: Allows Java programmers to write re-usable tests for web applications that, unlike HttpUnit, “drive” the actual web browser on the actual platform they intend to support. Tests can be leveraged for functional, stress, reliability. Requirement: Microsoft, OS Independent, Linux
• 
• Expect: A Unix automation and testing tool, for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, ssh, tip, etc. And by adding Tk, you can also wrap interactive applications in X11 GUIs. Requirement: Windows / UNIX
• Frankenstein: A Functional Testing tool for Swing applications. Frankenstein’s focus is on readable, simple, fast functional tests that can be shared and run by everyone on a team. Apart from automating your functional tests, you could also use Frankenstein for recording bugs so that they may be easily reproduced later. Requirement: JDK 1.4+
• FireWatir: Has a similar API to Watir, though accesses the DOM by invoking JavaScript by using the JSSh XPI to telnet into the browser. While Watir works with MSIE, FireWatir is compatible with Firefox 1.5 and above. FireWatir allows Watir scripts written for IE to work with Firefox as well, usually requiring either no change or very small changes to existing scripts. It is planned for FireWatir and Watir to be merged. The wiki includes info on compatibility issues between Watir and Firewatir.
• Funkload: Web functional testing and load testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF. Functional tests are pure Python scripts using the pyUnit framework.
• FWPTT: is a web application tester program for load testing web applications which can record normal and Ajax requests
• GITAK: TIBCO General Interface Test Automation Kit (GITAK) is a test-automation tool for Ajax applications. GITAK extends the Selenium Core test tool for Web applications. It lets developers create automated test cases and run scenarios to validate that an application is performing properly. Once a library of test cases has been built, the Ajax applications and changes to them can be tested with the push of a button. Requirement: IE 6, IE7, Firefox 2, Firefox 1.5.x
• GNU/Linux Desktop Testing Project: Aimed at producing high quality test automation framework and cutting-edge tools that can be used to test GNU/Linux Desktop and improve it. It uses the “Accessibility” libraries to poke through the application’s user interface. The framework has tools to generate “AppMap” by reading through the user interface components of an application. The framework also has tools to record test-cases based on user-selection on the application. GNU/LDTP can test any GNOME application which are accessibility enabled, Mozilla, OpenOffice.org, any Java application (should have a UI based on swing) and KDE 4.0 applications based on QT 4.0 (based on the press releases by KDE). Requirement: Linux
• Harness: An open source Java API for creating Java test software
• HtmlUnit: Java unit testing framework for testing web based applications. (Similar in concept to httpunit but is very different in implementation) HtmlUnit models the returned document so that you can deal with pages, forms and tables.
• httest: Scriptable HTTP Test Tool for testing and benchmarking web application and HTTP server development. Can act as client (requesting) and server (back-end for reverse proxys). Pattern matching answers (both server(s)and client(s)) to test validity. Has a very simple but powerful syntax. Can execute and stream shell commands into the HTTP stream and vice versa. Requirement: linux, solaris
• HTTPUnit: Java API for testing web sites without a browser.
• IdMUnit: Leading xUnit automated testing framework for Identity Management that simplifies and accelerates the functional testing of the solution. Test cases are defined and implemented in spreadsheet format. This product plugs into Eclipse. Requirement: Cross-platform
• IeUnit: A simple framework to test logical behaviors of web pages, released under IBM’s Common Public License. It helps users to create, organize and execute functional unit tests. Includes a test runner with GUI interface. Implemented in JavaScript for the Windows XP platform with Internet Explorer.
• iMacros for Firefox: Free Firefox add-on to record and automate web interactions. Can use variables inside the macros, and import data from CSV files. Includes user agent switcher, PDF download and Flash, ad and image blocking functions. The recorded macros can be combined and controlled with Javascript, so complex tasks can be scripted. The EXTRACT command enables reading of data from a website and exporting it to CSV files. Full Unicode support and works with all languages including multi-byte languages such as Chinese. STOPWATCH command enables capturing of web page response times
• Imprimatur: A web application functional testing tool. The tests are described in a simple XML file. Imprimatur handles HTTP methods, authentication and file uploads. The responses can be validated using regular expressions. Requirement: Java
• ItIN – Infopath testing in .Net: A framework for the testing of InfoPath forms. It is derived from the WatiN testing framework which is used for testing web applications. ItiN needs Visual Studio 2005 and InfoPath 2003 with the .Net programability support installed to work. You may have some trouble with the references, but it should be OK. Requirement: Windows
• ITP: Lightweight, yet powerful web application test harness. Test scripts written in XML. No programming required and no changes required to your web application. Supports sessions/cookies, POST form data. Command line based for integration into other tools. Also useful for regression and smoke testing.
• ItsNat, Natural AJAX: A Java AJAX web framework with functional web test built-in. Simulates a Universal Java W3C Browser in the server, the client DOM tree is basically a clone of the server and is updated automatically when the server changes usually as the response of an AJAX event. The server can fire W3C DOM events and send them to the browser simulating user actions. These are received again by the server as in a normal AJAX app. As the test code is in the server too, can check the expected GUI changes (checking the server DOM tree) or the expected business behavior (added/removed/updated data). Requirement: Any supported platform by Java VM 1.4 or upper
• ivalidator: Regression testing framework written in java but by no means restricted to java testing. Test suites are declared in XML. Especially designed for complex testing scenarios and integration testing. Requirement: JDK 1.3
• Jacobie: A Java API for use with Internet Explorer. Based on the JACOB project (Java to Com Bridge) and the IE COM Object, it directly controls IE from java. This API can be used as a true end-user web browser test with IE and not a Http-Based test such as HttpUnit. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)
• Jameleon: A plug-in driven automated testing tool that separates applications into features and allows those features to be tied together independently, creating test cases. Test cases can be data-driven and executed against different environments and test case docs are generated from the test cases. The goal is to create an automated testing tool that can be used for the enterprise. A UI that ties a series of features to a test case, generating both the test script and the test case documentation is in the works. Requirement: OS Independent, JDK 1.4 or higher
• jDiffChaser: A GUI comparison tool that automates diffs detection between versions. You can record and play scenarios on two different releases of the same Swing application (in sequential or parallel mode); jDiffChaser compares both screens, shows you the differences and list them in a report with images highlighting the diffs. Requirement: Linux, OS X, WinXP
• Jemmy: A tool allowing you to create automated tests for Java GUI applications. Tests are written on Java, using Jemmy as a regular Java library. No recording (yet), no GUI, no XML, no bells and whistles – all the work being done in Jemmy is dedicated to test stability, so it could be used for big, complicated and “dynamic” Java GUI applications. Requirement: Java 1.2 or above
• JFunc: JUnit Functional Testing Extension: An extension to the JUnit testing framework to make it easier for use with functional tests. Functional testing (also called integration testing) significantly differs from unit testing in a number of respects. Part of this project is dedicated towards putting together code to address these differences; the other part of this project is putting together methodologies for functional testing. Requirement: JUnit
• JSystem: An open source framework for writing and running automated system testing. JSystem includes: 1. Services Java API – exposes JSystem services 2. JSystem Drivers- Java modules used to interfaces with the system under test. 3. JRunner – GUI application interface used for creating and running tests scenarios. 4. JSystem Agent – Execution engine used to run scenarios on a distributed setup. 5. JSystem Eclipse plug-in – accelerates the development environment setup and enforces JSystem conventions. JSystem is based on JUnit (tests and steps) and Ant (execution engine).
• jWebUnit: A Java framework that facilitates creation of acceptance tests for web applications. jWebUnit provides a high-level API for navigating a web application combined with a set of assertions to verify the application’s correctness. This includes navigation via links, form entry and submission, validation of table contents, and other typical business web application features. This code utilizes HttpUnit behind the scenes. The simple navigation methods and ready-to-use assertions allow for more rapid test creation than using only JUnit and HttpUnit. Requirement: OS Independent
• Latka: A functional (end-to-end) testing tool. It is implemented in Java, and uses an XML syntax to define a series of HTTP (or HTTPS) requests and a set of validations used to verify that the request was processed correctly. Requirement: JDK 1.3 or better.
• Linux Test Project: A collection of tools for testing the Linux kernel and related features. Our goal is to improve the Linux kernel by bringing test automation to the kernel testing effort. Requirement: Linux
• LogiTest: The core application in the LogiTest suite. The LogiTest application provides a simple graphical user interface for creating and playing back tests for testing Internet-based applications. Requirement:JDK 1.2 or higher
• LReport: Command line tools for comparing csv files and databases (on the level of particular selects). The tools also support test documentation by nice formatting of selects’ results. Requirement: Tested on Win32 but should work on other platforms
• Mactor: An extensible tool for system integration testing. It can facilitate tests of any XML-based integration regardless of the type of message transfer protocol used (HTTP, SOAP, file-system and IBM MQ series are currently supplied with the tool)
• Marathon: A general purpose tool for both running and authoring acceptance tests geared at the Java Platform Version 1.3 or later. Included with marathon is a rich suite of components to help you interact with your application at the User Interface Level (GUI). To aid with the regression testing of existing applications, Marathon comes bundled with a recorder to capture events as you use and interact with your application. These events are then converted into a valid Marathon test which can subsequently be played back. Requirement: Java 1.3 or later
• 
• MaxQ: A free web functional testing tool. It includes an HTTP proxy that records your test script, and a command line utility that can be used to playback tests. The paradigm of MaxQ is similar to commercial web testing tools like Astra QuickTest or Empirix e-Test. These products are quite expensive. MaxQ hopes to provide the essential features: HTTP test recording, scripting, and playback without the huge cost.Requirement: Java 1.2 or later
• Mechanize: Ruby library for automating interaction with websites; automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Also keeps track of the sites visited. NOTE: does not handle javascript.
• Mockito: Java mocking is dominated by expect-run-verify libraries like EasyMock or jMock. Mockito offers simpler and more intuitive approach: you ask questions about interactions after execution. Using mockito, you can verify what you want. Using expect-run-verify libraries you are often forced to look after irrelevant interactions. Mockito has very slim API, almost no time is needed to start mocking. There is only one kind of mock, there is only one way of creating mocks. Just remember that stubbing goes before execution, verifications of interactions go afterwards. Requirement: Java
• MozUnit: Develop test-first style or just test against regressions: MozUnit provides framework, test runner, source browser, and API hooks for personalized reports. MozUnit is part of MozLab, a suite of tools and libraries for developers of AJAX and Mozilla applications, packaged as a Firefox extension.Requirement: Firefox
• OLVER – Open Linux VERification: A test suite for automated conformance and functional testing of various Linux distributions against LSB standard requirements on base system interfaces behavior. The tests are being developed at the Linux Verification Center of Russia. Requirement:Linux
• org.tigris.mbt: An implementation of Model-based testing built in Java. It allows you to generate test sequences from a finite-state machine (graph). The test sequences can be created statically, or run dynamically.Requirement: Any platform that runs Java 1.4.2
• Ottomate: Suite of six Mac OS X Automator Actions that contains everything needed to graphically configure automated, repeatable user-acceptance tests for web-based applications. Requirement: Safari
• PAMIE: ‘Python Automated Module For Internet Explorer’ Allows control of an instance of MSIE and access to it’s methods though OLE automation . Utilizes Collections, Methods, Events and Properties exposed by the DHTML Object Mode Requirement:Windows NT/2000
• Pounder: A utility for testing Java GUIs. It allows developers to dynamically load components, record scripts, and then use those scripts in JUnit. It supports custom components, drag and drop, and the examination of test runs in source. This project is no longer being actively developed. For similar tools under active development, the Pounder team recommend considering Abbot, Marathon, jfcunit and others. Requirement: OS Independent
• PureTest: From Minq Software AB, includes an HTTP Recorder and Web Crawler. Create scenarios using the point and click interface. Includes a scenario debugger including single step, break points and response introspection. Supports HTTPS/SSL, dynamic Web applications, data driven scenarios, and parsing of response codes or parsing page content for expected or unexpected strings. Includes a Task API for building custom test tasks. The Web Crawler is useful for verifying consistency of a static web structure, reporting various metrics, broken links and the structure of the crawled web. Multi-platform – written in Java.
• pywinauto: A python package that allows you to automate the windows GUI. Very easy to get started, and quite powerful. Requirement:Windows 2000, XP, +
• QAT (Quality Assurance Tests): Developed to ease the issues encountered by having to perform Quality Assurance tests across a variety of hardware and software combinations. The QAT tool can be divided into two main sections, the Agent, responsible for actually running each test or group of tests, and the Harness, which is responsible for test selection, management, result and agent co-ordination. Requirement:Java 2
• QMTest: CodeSourcery’s QMTest provides a cost-effective general purpose testing solution that allows an organization to implement a robust, easy-to-use testing program tailored to its needs. QMTest’s extensible architecture allows it to handle a wide range of application domains: everything from compilers to graphical user interfaces to web-based applications. Requirement: QMTest works with most varieties of UNIX, including GNU/Linux, and with Microsoft Windows.
• Rasta: A keyword-driven test framework using spreadsheets to drive testing. It’s loosely based on FIT, where data tables define parameters and expected results. The spreadsheet can then be parsed using your test fixtures.Requirement:Windows, Ruby
• Robot Framework: Robot Framework is a Python-based keyword-driven test automation framework for acceptance level testing and acceptance test-driven development (ATDD). It has an easy-to-use tabular syntax for creating test cases and its testing capabilities can be extended by test libraries implemented either with Python or Java. Users can also create new keywords from existing ones using the same simple syntax that is used for creating test cases.
• safariwatir: The original Watir (Web Application Testing in Ruby) project supports only IE on Windows. This project aims at adding Watir support for Safari on the Mac. Requirement: OS X running Safari
• Sahi: An automation and testing tool for web applications, with the facility to record and playback scripts. Developed in Java and JavaScript, it uses simple JavaScript to execute events on the browser. Features include in-browser controls, text based scripts, Ant support for playback of suites of tests, and multi-threaded playback. It supports HTTP and HTTPS. Sahi runs as a proxy server and the browser needs to use the Sahi server as its proxy. Sahi then injects JavaScript so that it can access elements in the webpage. This makes the tool independant of the website/ web application. Requirement:Needs Java 1.4+
• Samie: S.A.M. for I.E. is a Perl module (SAM.pm) that allows a user to run automated tests for their browser applications. Requirement: Windows NT/2000
• Scalable Test Platform: STP is a system for automating the QA testing process for the Linux Kernel, as well as automating benchmarking and regression testing on diverse hardware systems. Requirement: Linux
• Siege: http regression testing and benchmarking utility. It was designed to let web developers measure the performance of their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It allows the user hit a web server with a configurable number of concurrent simulated users. Those users place the webserver “under siege.”
• Selenium: Testing tool for browser-based testing of web applications. It can be used both for functional, compatability (it has extensive cross-browser support) and regression testing Requirement: Windows, Linux or Mac
• Selenium Grid: An open source web functional testing tool that can transparently distribute your tests on multiple machines to enable running tests in parallel, cutting down the time required for running in-browser test suites. This enables speed-up of in-browser web testing. Selenium tests interact with a ‘Selenium Hub’ instead of Selenium Remote Control. The Hub allocates Selenium Remote Controls to each test. The Hub is also in charge of routing the Selenium requests from the tests to the appropriate Remote Control as well as keeping track of testing sessions. Requires Java 5+ JDK, Ant 1.7.x
• SharpRobo: A Functional Testing and Recording tool for WinForm applications written in C#. It supports all the standard WinForm controls. SharpRobo records the tests in FIT format which can be played back using Fit (File or Directory Runner). Requirement:Windows NT/2000/XP
• SimpleTest: Unit testing framework which aims to be a complete PHP developer test solution. Includes all of the typical functions that would be expected from JUnit and the PHPUnit ports, but also adds mock objects; has some JWebUnit functionality as well. This includes web page navigation, cookie testing and form submission.
• soapui: A java-swing based desktop application for inspecting, invoking and functional testing of webservices over HTTP. It is mainly aimed at developers/testers providing and/or consuming webservices (java, .net, etc). Functional testing can be done interactively in soapui or within a CI-process using the soapui maven plugin.Requirement: Java 1.5
• Software Automation Framework Support (SAFS): Provides for the implementation of compatible keyword-driven test automation frameworks. Currently, developing independent, multi-platform, Java-based Driver. Will be followed by independent, multi-platform Engines. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)
• Software Testing Automation Framework (STAF): An open source, multi-platform, multi-language framework designed around the idea of reusable components, called services (such as process invocation, resource management, logging, and monitoring). STAF removes the tedium of building an automation infrastructure, thus enabling you to focus on building your automation solution. STAX is an execution engine which can help you thoroughly automate the distribution, execution, and results analysis of your testcases. STAX builds on top of three existing technologies, STAF, XML, and Python, to place great automation power in the hands of testers. STAX also provides a powerful GUI monitoring application which allows you to interact with and monitor the progress of your jobs. Requirement:Windows, Linux, Solaris, AS/400, AIX, HP-UX, Irix
• Solex: This project is a set of Eclipse plugins providing non regression and stress tests of Web application servers. Test scripts are recorded from internet browser thanks to a built in web proxy. Requirement: Eclipse 2.1 or above
• SWAT (Simple Web Automation Toolkit): A library written in C# designed to provide an interface to interact with several different web browsers. SWAT also includes components to integrate with Fitnesse allowing Q/A engineers to automate web application testing. Requirement: Windows (IE and FireFox)
• SWTBot: A functional testing tool for SWT and Eclipse applications. The focus of SWTBot is to provide a simple, readable and fast way to write tests. The API is simple which means that everyone on a team can use SWTBot to write functional tests. It is also very flexible when it comes to extensibility. Requirement: SWT/Eclipse
• Systin: Systin stands for System Testing in .Net and allows you to write system-level tests in a “domain language”. This is a port of the popular Systir program. Systin will allow for an abstraction of Test Case specification and Test Case automation execution. Requirement: .Net Windows
• tclwebtest: A tool for writing automated tests on web applications in tcl. It implements some basic html parsing functionality to provide comfortable commands for operations on the html elements (most importantly forms) of the result pages.
• TestGen4Web: A capture-replay tool which can record user actions on Firefox, saving the recording to an XML file, and replaying the saved recording. The output of the recorder can also be translated into automatic testing scripts such as httpunit, selenium, simple-test, etc. Requirement: Firefox 1.5 alpha1 +
• TextTest: An application-independent tool for text-based functional testing. This means running a batch-mode binary in lots of different ways, and using the text output produced as a means of controlling the behaviour of that application.Requirement: Most UNIX flavours + Windows XP (not Windows 9x)
• Tomato: (the Automation Tool Abstraction Project) An abstraction layer for automation engines. Its design allows automation scripts or tests to be written in one language, against one library, and remain portable across different architectures, OS platforms, and even widely different automation engines (e.g. HP Mercury Interactive WinRunner or the Linux Desktop Test Project). Requirement: Windows/Linux
• Toster – The Object-oriented Sofware Testing Environment: A system for sharing a set of tools that allow you to implement methods for object-oriented testing. Any method based on UML diagrams and on the software source code can easily be implemented as a TOSTER module. The environment itself makes a number of mechanisms available, such as information transfer from UML diagrams, mapping this information to source code, introducing modifications to the source code, launching the tested application, or presenting the results.
• Watij: (pronounced wattage) stands for Web Application Testing in Java. Based on the simplicity of Watir and enhanced by the power of Java, Watij automates funtional testing of web applications through the real browser. There is a Google group at http://groups.google.com/group/watij Requirement:Windows
• WatiN: WatiN stands for Web Application Testing in dotNet. Inspired by Watir, WatiN enables web application testing, through Internet Explorer on a Windows platform, expressed in any .Net language.Requirement: Windows
• Watir: Watir (Web Application Testing in Ruby) is a functional testing tool for web applications. It supports tests executed at the web browser layer by driving a web browser and interacting with objects on a web page. It uses the Ruby scripting language. Requirement: Windows (currently only supports Internet Explorer)
• WebCorder: Free GUI web testing tool from Crimson Solutions, developed in VB. Designed for end users who are doing web based software testing, as a simple tool to record test scenarios, and play them back and generate log files. The user may also check for text or images on the screen or save screenshots.
• Web Form Flooder: A Java console utility that will analyze a Web page, complete any forms present on the page with reasonable data, and submit the data. The utility will also crawl links within the site in order to identify and flood additional forms that may be present.
• WebDriver: A developer focused tool for automated testing of webapps: WebDriver has a simple API designed to be easy to work with and can drive both real browsers, for testing javascript heavy applications, and a pure “in memory” solution for faster testing of simpler applications. Requirement: Any java-compatible platform
• WebInject: A free tool for automated testing of web applications and services. It can be used to test any individual system component with an HTTP interface, and as a test harness to create a suite of automated functional and regression tests. Requirement: Windows, OS Independent, Linux
• Webrat: Ruby-based utility to enable quick development of web app acceptance tests. Open source by Bryan Helmkamp. Leverages the DOM to run tests similarly to in-browser test tools like Watir or Selenium without the associated performance hit and browser dependency. Best for web apps that do NOT utilize Javascript; apps using Javascript in-browser tools may be more appropriate.
• WebTst: AWeb development test infrastructure. It aims to simplify testing by implementing a capture engine: a Web proxy which records a tester’s actions using a real browser, and then replays them during testing. It comes with support for digital certificates, and a number of simple tests, such as cookie setting, pattern matching, response status, and many others. It features an extensible plug-in system. Requirement: POSIX, Linux
• WET: An opensource web automation testing tool which uses Watir as the library to drive web pages. You don’t have to download / install Watir separately or know anything about Watir. WET drives an IE Browser directly and so the automated testing done using WET is equivalent to how a user would drive the web pages. WET allows you to perform various checks as a part of the testing process by using Checkpoints. Requirement: Windows 98/ME/2000 SP3/XP SP2/Server 2003
• Win32::IEAutomation: A Perl module which automates functional testing of web applications. It can be used to automate any complex web application including dynamic frames and popup windows. It is an object oriented module and all methods are like user actions on web browser. Requirement: Windows (only Internet Explorer is supported)
• XML Test Suite: Provides a powerful way to test web applications. Writing tests requires only a knowledge of HTML and XML. We want XmlTestSuite to be adopted by testers, business analysts, and web developers who don’t have a java background. Requirement: Windows 95/98/2000, Windows NT/2000, Linux, SunOS/Solaris
• Yawet: Visual web test tool from InforMatrix GmbH enables graphical creation of web app tests. Create, run and debug functional and regression tests for web applications. Can verify HTML, XML, and PDF’ ability to do report generation, reusable step libraires and parameterization. Freeware; download jar file and start by double-click or with command javaw -jar yawet.jar

Please comment below on your thoughts/experience on any of the tools listed above and/or if you know of any other tools that should be added.

Related posts:

1. 15+ Open Source Test Management Tools
2. 100+ Open Source/Free Security Tools
3. 10+ Open Source Link Checking Tools

I attended the Conference of the Association of Software Testing (CAST) this week, which is hosted by the Association for Software Testers (AST). I was very impressed by not only the topics but also the great people. It was nice being able to sit down and discuss topics that we all face out in the field (lack of requirements, process, communication, etc). There were tutorials on different topics like metrics and agile development.

The conference had tons of great speakers. I didn’t get to go to all of the presentations but the ones I did go to were loaded of great info. During each presentation there was a time called “open season” where the attendees could ask the experst questions/speak their mind, etc. It was nice being able to hear and discuss the way people are impacted by the topics today and how they will use the newly learned information.

I think everyone within the QA field should join AST, not only for the wonderful conference but also for the learning experience.

The AST also offers it’s members BBST training. The training is online and taught by other AST members.

Even if you have knowledge on a specific topic there is always more to learn. Why not learn from others in the field?

Related posts:

1. 30+ Common Testing Mistakes
2. 100+ Open Source/Free Functional Testing Tools
3. 15+ Open Source Test Management Tools

No matter how good you think your code is there is always going to be a bug/defect of some kind. Defect numbers may range from 1 to thousands so it’s important to have a very good tracking tool. There are commercial tools out there but I figured I’d focus on the open source tools.

Below are some open source tracking tools that you can use for your projects to track bugs/defects, issues, enhancements and much more.

• Abuky: Abuky stands for the Aoo BUg tracKing sYstem, while AOO stands for Art Of Open Source. Abuky is a system for tracking bugs and aiding the developer to fix them, written in Java with JSP as web interface. Requirement: Linux, Windows, Solaris
• Anthill Bug Manager: A tool that aids code development by keeping track of bugs in a multi-project, multi-developer environment. It accomplishes this with a clean, simple, and fast interface that contains all the essential features
  but avoids the enormous complexity associated with most other projects of this type. Requirement: OS Independent
• BTsys: A lightweight desktop bug tracking system, written using C# and ADO.NET for small developer teams. Easy to install and learn how to use. Requirement:All 32-bit MS Windows (95/98/NT/2000/XP)
• Bug-A-Boo: A web based bug reporting system for Linux. It runs on any web server providing CGI
  functionality. Bug-A-Boo does not need any database server but brings along its own local tables it accesses with tdbengine.Requirement: Linux
• BugBye: A web-based bugtracking system developed using ASP.NET technology and C# as scripting language. It offers all the features needed to manage improvement, bugs, and so on. It also provide statistics help-desk management, and further options which allow the user to simplify all the stage of project development and maintainance. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)
• Bugfree: A lightweight and simple web-based bug tracking system available in Chinese and English. Requirement: Windows
• Buglog: Allows you to easily record bugs into a database along with description and screenshots of the bugs (file upload feature). The BLT allows tracking of all bugs on a project-wise basis thus making it a bug logging and tracking solution. BuglogV2 incorporates a powerful search feature allowing developers to locate bugs effortlessly. Requirement: All 32-bit MS Windows(95/98/NT/2000/XP)
• BugNet: A web based bug / issue tracking application programmed in C# and asp.net. The main goals are to keep the codebase simple, well documented, easy to deploy and scalable. Major features include: Generic DAL / XHTML and CSS layout. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)
• BugRat: Java software that provides a sophisticated, flexible bug reporting and tracking system.
• BUGS – The Bug Genie: A web-based bug tracking tool, designed with ease of use as it’s foremost goal, but also features a high level of flexibility for developers and administrators. Requirement: PHP4 (or later) and MySQL.
• Bugs Online: Originally developed in 1997 to serve as the primary bug and issue tracking system to be utilized during a large development oriented project. The Bugs Online system is a very flexible and capable system for bug and issue tracking. Requirement: Windows NT 4.0 SP3+, MS IIS 3 w/ ASP
• BugtrackWeb based bug tracking system written in Perl/DBI. Supports multiple users, projects, components, versions and email notification. Requirement: Linux, Solaris, Windows
• BugTracker.NET: Web-based bug or issue tracker written using ASP.NET, C#, and Microsoft SQL Server/MSDE. It is in daily use by hundreds (maybe thousands) of development and support teams around the world. Requirement: All 32-bit MS Windows (95/98/NT/2000/XP)
• Bugzilla: Features include: integrated, product-based granular security schema, inter-bug dependencies and dependency graphing, advanced reporting capabilities, a robust, stable RDBMS back-end, extensive configurability, a very well-understood and well-thought-out natural bug resolution protocol, email, XML, console, and HTTP APIs, available integration with automated software configuration management systems, including Perforce and CVS (through the Bugzilla email interface and checkin/checkout scripts), too many more features to list.
• CodeTrack: Bug database with a friendly web front end aimed at medium and small development shops. Particularly suited for intranet and extranet environments, CodeTrack includes built-in strong authentication, and allows custom access control to individual projects. No database is required as bug data and developer notes are stored using simple XML text files. Requirement: Apache and PHP
• Debian bug tracking software: The Debian bug tracking system is a set of scripts which maintain a database of problem reports. Requirement: Unix
• DITrack (Distributed Issue Tracker): Implemented in Python and runs in UNIX (*BSD, Linux, MacOS X) and Windows environment.
• Ditz: Open source distributed issue tracker designed to work with distributed version control systems written in Ruby
• eTraxis: A web-based bug tracking system with unlimited number of custom workflow templates. Some of the main features are: fully custom templates, advanced filters, LDAP support, email notifications, subscriptions, reminders, flexible permissions management, graphical project metrics, etc. Requirement: OS independent



• Eventum: A user-friendly and flexible issue tracking system that can be used by a support department to track incoming technical support requests, or by a software development team to quickly organize tasks and bugs. Requirement: PHP, MySQL
• Flyspray: Originally developed for the Psi project, Flyspray is an easy to use bug tracking system for those who do not require all the complexities of something like Bugzilla. It supports multiple users, file attachments, and Jabber notifications. Requirement: Web Environment
• FOSSology: Started out as an internal development effort at Hewlett Packard Company (HP).
• Gjallar: Highly customizable issue tracker with advanced email integration and ability to work offline and synchronize with server. Requirement: Windows
• GNATS: A portable incident/bug report/help request-tracking system which runs on UNIX-like
  operating systems. It easily handles thousands of problem reports, has been in wide use since the early 90s, and can do most of its operations over e-mail. Several front end interfaces exist, including command line, emacs, and Tcl/Tk interfaces. There are also a number of Web (CGI) interfaces written in scripting languages like Perl and Python. Requirement: OS Independent
• Helis: Iincludes the main features of most bug tracking systems. Requirement:Linux web server (php 4/mysql + cgi)
• Issue Tracker: by ASP.NET. Issues can be tracked, related issues can be displayed & more. Issue Tracker supports MS Access & MSSQL for storing the data.
• Issue Tracker Product: A straight forward and user friendly web application built on top of the Zope application server. Requirement: OS Independent, Zope
• itracker: An issue tracking system designed to support projects with independent user bases. Features include multiple versions and components, detailed issue histories and email notifications. Soon based on Java Spring 2.0 and Hibernate. Requirement: OS Independent
• JitterBug: A web based bug tracking system. JitterBug operates by receiving bug reports via email or a web form. Authenticated users can then reply to the message, move it between different categories or add notes to it. In some ways JitterBug is like a communal web based email system. This web page is itself a JitterBug page.
• jTrac: A generic issue-tracking web-application that can be easily customized by adding custom fields and drop-downs. Features include customizable workflow, field level permissions, e-mail integration, file attachments and a detailed history view. Requirement: OS Independent
• Kwok Information Server: IT web application, providing a centralized application for managing IT assets, software licenses, service contracts, issues, and contacts. Additional modules include portal, RSS, and blogs.
• Lighthouse Pro: ColdFusion bug tracking application. It lets you easily track bugs and issues for a project. With Lighthouse Pro, you can monitor the complete life cycle of an issue, from creation to QA to closure.
• Mantis: A php/MySQL/web based bugtracking system. Requirement: Windows, MacOS, OS/2, and a variety of Unix operating systems. Any web browser should be able to function as a clientWindows, MacOS, OS/2
• Ohloh: Is more than just a tracking database
• oops-easytrack: A Bug Tracking System, implemented as a LAN-based and a web-based version. Bugs are created, modified, logged or searched. Users, projects,components and releases can be administered. Reports of 4 types may be generated. Requirement: OS Independent
• Open Track: A problem tracking(PR) system that is table driven and easily
  configurable/customizable for a variety of PR applications. Project defect tracking, help desk tracking, and requirements gathering can be easily handled by OpenTrack.
• PEST: A bug tracking system written especially for a web enviroment. It supports good testing and bug tracking processes, as well as notification.
• phpBugTracker: Provides a codebase that is independent of the database and presentation layers. Requirement: webserver with PHP 4.1.0+
• Project Dune: Mostly a web-based issue tracker with integrated modules: Cocomo II estimates, Scrum tasks, timesheet, simple customer management and a browser-based document writer. Requirement: OS Independent
• Project Open: Web-based ERP/Project Management software for organizations with 2-2000 users
• RadTracker: A MYSQL-PHP web based issue tracking system built for healthcare but generic enough for most needs. Designed to ease the support of multiple idiosyncratic information systems needing their own knowledge base and support mechanism.
• Redmine: Flexible project management web application. Written using Ruby on Rails framework, it is cross-platform and cross-database. Requirement: Ruby and Ruby on Rails must be installed.
• Request Tracker: RT is an industrial-grade ticketing system. It lets a
  group of people intelligently and efficiently manage requests submitted by a community of users. RT is used by systems administrators, customer support staffs, NOCs, developers and even marketing departments at over a thousand sites around the world. Requirement: Written in object-oriented Perl, RT
  is a high-level, portable, platform independent system
• Roundup Issue Tracker: Roundup is a simple-to-use and -install issue-tracking system with command-line, web and e-mail interfaces. It is based on the winning design from Ka-Ping Yee in the Software Carpentry “Track” design competition.
• Scarab: The goal of the Scarab project is to build an Issue / Defect tracking system that has the following features: A full feature set similar to those found in other Issue / Defect tracking systems: data entry, queries, reports, notifications to interested parties, collaborative accumulation of comments, dependency tracking In addition to the standard features, Scarab has fully customizable and unlimited numbers of Modules (your various projects), Issue types (Defect, Enhancement, etc), Attributes (Operating System, Status, Priority, etc), Attribute options (P1, P2, P3) which can all be defined on a per Module basis so that each of your modules is configured for your specific tracking requirements. Built using Java Servlet technology for speed, scalability, maintainability, and ease of installation. Import/Export ability via XML allowing for easy migration from other systems (like Bugzilla). Modular code design that allows manageable modifications of existing and new features over time. Fully customizable through a set of administrative pages. Easily modified UI look and feel. Can be integrated into larger systems by re-implementing key interfaces.
• Stabilizer: The Stabilizer bugtracking system aims to quickly stabilize buggy GUI applications so that people can get real work done with them. Users collaboratively and quickly stabilize a buggy GUI application simply by using the application normally and reporting any bugs that they encounter. As soon as a few people report the same bug, warnings will be issued to all users whenever they are about to trigger that bug and they will be given the opportunity to abort the input event — thus avoiding the bug altogether and keeping the application stable. Requirement: All POSIX (Linux/BSD/UNIX-like OSes), Linux
• The Bug Genie: Built with PHP, has multilanguage support & provides an easy to use interface.
• Trac: An enhanced wiki and issue tracking system for software development projects. Trac allows wiki markup in issue descriptions and commit messages, creating links and seamless references between bugs, tasks, changesets, files and wiki pages. A timeline shows all project events in order, making getting an overview of the project and tracking progress very easy. Requirement:Python, CGI-capable web server
• Track+: A web based issue tracking and project management application targeted at medium to large commercial development projects. Track+ can be used instantly, yet it is flexible and configurable. Track+ is easy to use, provides user centric as well as project centric views, has configurable workflows, supports e-mail submission of issues, and has a comprehensive time and cost tracking module. Track+ is free to academic institutions, open source projects, and charities, and generally free to anybody up to 10 active users in the database. Track+ comes with the complete source code. Requirement: OS Independent (Written in an interpreted language)
• TrackIt: A Web-based project tracking tool that incorporates defect tracking functionality. It is designed from the ground up to provide maximum flexibility, customization, and most importantly, usefulness to the developer. It has built-in support for various Extreme Programming constructs, as well as full CVS and Subversion integration. It also supports simple listings via HQL and advanced reporting via SQL. Requirement: JRE 1.5
• WebIssues is an open source, multi-platform system for issue tracking and team collaboration. The server can be installed on any host with PHP and MySQL, PostgreSQL, Firebird or SQL Server. The client is a native desktop application for both Windows and Linux.
• WREQ: Designed to be a distributed request/problem tracking system with builtin knowledge database to help systems personnel to stay on top of requests and to prompt knowledge sharing among all local support groups. Requirement: To use wreq, first you must have perl version 5 with GDBM support installed on your web server.
• zenTrack: Highly configurable bug tracking, project management, and help desk solution. Project focus is on configurability, usability, and clean code. Requirement: OS Independent

Please comment below on your thoughts about any of the tools listed above and/or if you know of any other tools that should be added.

Related posts:

1. 15+ Open Source Test Management Tools
2. 10+ Open Source Link Checking Tools
3. 100+ Open Source/Free Functional Testing Tools

It is very important to make sure that there are no broken links on your site. A link may be active when your first added it but as the days, months, years go by the link may end up being broken.

Below are some open source tools that can help you find those broken links.

• Bugkilla – J2EE Functional Test Suite: Bugkilla will be a set of java tools for the functional test of J2EE Web Applications. Specification and execution of tests will be automated for web front end and business logic layer.
• DLC (Dead Link Check): It can generate an HTML output for easy checking of the results, and can process a link cache file to hasten multiple requests (links life is time stamp enforced). Written in Perl
• ht://Check: Outputs a simple report. It can retrieve information through HTTP/1.1 and store them in a MySQL database. Most of the information is given by the PHP interface which comes with the package and that is able to query the database built by the htcheck program. Requirement: You need a Web server to use it, and PHP with the MySQL connectivity module.
• InSite: A site management tool written in Perl. Requirement: Linux. Requires libwww and MIME::Lite (available at any CPAN mirror).
• Jenu: A multithreaded, Java 1.3 (swing) based Web site URL Link checker. It’s a copy of a nice multi-threaded link checker for the PC called Xenu. Requirement: Java 2 (1.3) runtime.
• JSpider: A Web spider engine. It is a robot that will generate web traffic, just like you would do when you are browsing the internet. You can control and configure the robot’s behaviour to adapt it to your needs. On it’s way through sites, it will gather all kinds of information you might be interested in. You can use a web spider for different purposes: searching dead links (404 errors) on your website, testing your site’s performance under havy load, copying an entire site to your harddisk, etc … Requirement: Linux, Solaris, Windows, and other Java-enabled Operating Systems.
• LinkChecker: A link management solution integrated into Plone. Requirement: Plone 2.0.5, 2.1, and 2.5 beta



• Linklint: Perl program that has the ability to check local-file and HTTP site checking. Creates a report of which URLs have changed since the last check.
• Link Page Generator: Automatic link management program with -check option for marking/eliminating bad links (in cron job). Written in Perl.
• LinkVerify: Checks a set of hypertext files whether all references to external resources are valid. In HTML this applies mostly to hyperlinks and embedded images. Style sheets will be checked too. Requirement: Java 1.1 is required
• SourceForge – LinkChecker: With LinkChecker you can check your HTML documents for broken links Requirement: Python 2.2.1. For HTTPS support you need to compile Python with the SSL _socket module.
• W3C Link Checker: Checks that all the links in your HTML document are valid. There is a command-line interface and an online version. The Link Checker can easily be installed on one’s server
• Xenu’s Link Sleuth: checks Web sites for broken links. Link verification is done on “normal” links, images, frames, plug-ins, backgrounds, local image maps, style sheets, scripts and java applets. NOTE: This one is free but NOT Open Source

Please comment below on your thoughts about any of the tools listed above and/or if you know of any other tools that should be added.

Related posts:

1. 50+ Open Source Tracking Database Tools
2. 15+ Open Source Test Management Tools
3. 100+ Open Source/Free Functional Testing Tools