Written in collaboration with Efrat Wurzel

Got a new product to test, and the most documentation anyone can provide is an e-mail saying “wouldn’t it be cool if we got drunk and then wrote some code”? Don’t worry – testing without a spec is not quite the disaster you were expecting. Using some exploratory testing techniques and some stubbornness, you can reach excellent coverage of any product – and you might manage to have some fun with it.

Understanding the Context

Start by figuring out what the product does, and what it was intended to do. In a product without a spec, the difference between the intention and the implementation can be staggering. Use your common sense to understand the goal of the product (although, really, someone should be able to explain at least this much), and how it tries to achieve that goal.

Next, explore the product in very general terms. Figure out the work-flows and the main options. You don’t have to get bugged down in details yet.

Now research the field your product is in (photo editing, games, word processing, e-commerce, etc) and make a note of things you expect to see in the product. Certain basic concepts and actions are bound to be relevant. Explore other products in the field and see what expectations users might have developed.

Exploring the Product

At this stage you have two goals: to discover how the product works, and to start planning testing. A bonus of this stage is that it will reveal a lot of bugs. A very large lot of bugs.

To achieve your goals you need to answer a lot of questions; any question that comes to mind. Don’t assume a question is too big or too small for this stage.

Before you start, you probably want to sort the product into sections, to help you organize your exploration. Pick whichever division you’re most comfortable with – by window, entity, work-flow or anything else. It’s up to you, and it’s important that you’re comfortable with your sorting method and can always tell where you are within it.

Your own Q&A

Some questions you can ask yourself about your product (this is by no means an exhaustive list):

Beyond the basic work-flow:

1. Who are the main entities in the product? What are their characteristics? Which of those characteristics is mandatory?
2. What can generate an error, and how are errors handled? How are they communicated to the users? Do you have access to logs?
3. Are the work-flows one-way or can the user undo? How many actions can be undone? Which actions can never be undone?
4. Are there multiple input and output types? Are there limitations on inputs and outputs?
5. How do you handle missing information? How do you handle corrupt files or failed processes?

Communication:

1. Does the product require an internet connection? How do you connect to the internet (Ethernet, 3G, WiFi)? What’s your network usage, and does it match what commonly used networks can support? How do you handle moving from one connection method to the next? How do you handle losing all connectivity?
2. How many servers do you use? Is there load-balancing? What services are on the server? What functionality is done on the servers, and what is done by the clients?
3. What’s in the post and what’s in the get? Do you use cookies? How long does a cookie live, and is it updated? What do you do if cookies are disabled?

General actions:

1. Do you accept payments? Which methods? Do you handle the payments, or do you direct the user to an embedded third-party payment processing section? What information do you store? What are the different states of services or goods that require payment?
2. Does the product have different states to maintain? Are they maintained in the database, or the session, or a cookie?
3. Are there configurations? Which can the user change, and which can only an administrator change? Which are only available in a configuration XML? Do you support exporting and importing configurations, or directly copying them?
4. Can users upload files? How? How can they view them? Can they edit or delete them? Which file types do you accept? Are there size limits? Can they download files?

Information:

1. Is there a database? What are the tables and columns? What are the indexes? What’s the rational behind these decisions?
2. Do you connect to third-party APIs? Do you offer an API to others? Do you use your own API? How do you query the APIs, how do they return information?
3. What’s done to information when it’s sent to the database, and what’s done when it’s sent back from the database? What manipulations on displayed information can users do? What’s editable, what can be entered by the user but never be edited, and what is system-generated?
4. How do you query the database? Are there stored procedures? How are results returned? Are there jobs that run only once a day, or once a week?
5. Are there XMLs? How many, and what’s the rational behind the number?

Interface:

1. What forms are available? Can users contact you? Can they contact each other? Can they send requests? How are forms validated?
2. Which languages do you support? Do you support right to left languages? Do you support languages with multiple input methods (like Japanese)?
4. Which input and interaction methods can be used (mouse, keyboard, voice, touch-screen)? Was the product built to support accessibility? Power-users?
10. If it’s a mobile application, does it use the GPS? The gyro? Does it have landscape and portrait modes? What does it do if the GPS is turned off, if the device doesn’t have a gyro, if landscape mode is locked etc?

Users:

1. How do users register? How many types of users are there? Can you suspend users? Can you delete them? Can they delete themselves? What do you do with their information?
2. Are there functions that are only available to logged in users?
3. How long does a log-in live? How are sessions identified, and how are they closed?
4. How much of a user’s history do you display? What can the user do with it?
5. Can users see each other’s information? How can users hide information?

Where and how does it run?

1. Where (other than where you’ve been working) can your product run? Windows, Linux, Mac, Android or iOS? If it’s browser-based, which browsers? Are there version limits? How does the product adjust to different operating systems or browsers?
2. How is the product installed? If it’s for mobile devices, which permissions does it require and what does it do with them? Are there dependencies? How are installations errors caught and displayed to the users?
3. What’s an acceptable memory usage? CPU usage? Is there a limit on how much storage you can use? What do you do when you run out of space?
4. What hardware do you need? Speakers, web cameras, microphones or special monitors?