During the planning stage of software development, one of the most important activities is understanding what problem you are trying to solve. While this seems simple, it’s more than just hearing a client or user tell us “I have problem X”. It’s up to us to dig deeper and understand a problem from many angles. More than that, we must ask “Is this really the problem”? Or are there underlying root causes of the problem that must be addressed? If we don’t do this, we risk solving the wrong problem or developing a solution that won’t work. For example, consider the following:

Now consider the following questions:

• What is the problem that the cities were trying to solve?
• What are the roots of that problem?
• Did the solution solve the problem?
• What problem did the solution solve?

This particular solution didn’t address the problem of homelessness; at least not in a way that truly eliminates it. Instead, the solution was to shift the problem from one that can be seen, to one that can’t be seen. By not “feeding the bears”, the solution was designed to force the homeless to move to another city or jurisdiction where they might be able to find food. The problem of homelessness was just moved from one spot to another.

Of course, this begs the question – how do we identify and solve the real problem?

Asking Good Questions

Identifying the real problem requires us to ask good questions that will allow us to immerse ourselves in the challenge, and to understand the problem in as much detail as possible.

Further, it requires us to take our time. Complex problems – such as broad social challenges – are interdisciplinary and multifaceted; so you likely won’t identify the real problem immediately. You’ll need to ask many questions, do a lot of research, and critically assess your assumptions. You’ll also need to consider which biases might get in the way of understanding the problem as completely as possible.

But what is a good question?

A good question allows us to gather the information we need as efficiently and accurately as possible, while not confusing the person providing the answer. Good questions are free of personal biases, judgements, and beliefs. Good questions are also free of language that might lead the respondent to provide an answer they think we want to hear.

This might sound trivial and simple, but developing good questions takes time, care, and a lot of thought. It also involves considering the type of question we are asking, when we are asking it, and how we are asking it.

In general, questions can be considered to be open or closed. They provide different types of data, and are useful in various situations, depending on what you are trying to achieve.

Open Questions

An open question allows the respondent to answer how they feel necessary. For example, Can you tell me about your childhood? is an open question. A respondent could provide a short answer, such as “it was great”, to a long and winding story describing seminal moments in their life.

Before continuing, consider:

• Why might you ask a respondent – such as our client – an open question?
• When might you want to ask the client this type of question?

Open questions allow us to gather information (often exploratory) about a topic we aren’t familiar, with while simultaneously demonstrating our respect for the respondent. In many cases, the information we collect is full of surprises and bits of details that lead us to things we didn’t know and might never have thought to ask about. This is particularly true in situations where we are asking open questions about a topic that is outside our experiences or disciplinary training.

Data collected with open questions are qualitative and may be more heavily weighted based on who provides the information. Data collected from experts in a particular domain will likely carry more weight than data from someone outside of that domain.

Analyzing data collected from open questions is more difficult than data collected in other ways. Qualitative data are noisy and subject to the respondent’s current emotional, physical, or mental state, biases, experiences, and personal contexts. It’s also much more difficult to identify trends in qualitative data than in responses to a simple survey.

Closed Questions

A closed question is typically written in such a way as to limit the potential responses that a client might provide. For example, the question Do you currently live in Guelph? likely will have only a yes or no response (although I’m not sure/I don’t know might also work). That’s not to say that a person who is asked this question won’t opt to provide you with more details, however, the question itself isn’t designed to invite more. Closed questions are typically what you see on multiple-choice exams or surveys.

Before continuing, consider:

• Why might you ask a respondent a closed question?
• When might you want to ask the client this type of question?

Closed questions allow us to collect information that is easily quantifiable or easily placed in bins. These data can be used to test for statistical significance, create groups/categories, or identify trends. However, they don’t allow us to necessarily capture the variation or nuance because they are asked with a specific set of responses provided to the respondent (such as is seen in a survey). This implies that we have a priori understanding or knowledge. Finally, closed questions don’t necessarily allow us to capture the why behind a response.

Open and closed questions are useful, but each has its time and place. In an exploratory setting, it’s probably better to focus your energy on open questions to learn as much as possible about an unfamiliar (such as our community partner’s challenge). Later, when you are in the midst of development, you’ll likely shift toward closed questions. That’s not to say that you won’t continue to use open questions later in development, nor is it to say that you won’t use closed questions when you first chat with the community partner, but the proportion of each will likely change as the project proceeds.

Things To Avoid

Once you’ve decided on the type of question you want to use to understand a problem (or to verify your understanding of a problem), you’ll also want to consider the following:

Other Things To Consider

When collecting information, you must provide the respondent with some preliminary information. This might include a description of why you are asking them questions, how their data answers/data might be used, how long their information will be stored, how it will be stored, who will have access to it, and more. This lets your respondent know that their responses are important and valued, and helps to also provide them with a sense of trust in you and your ability to use their data wisely and appropriately.

After you’ve asked each question, give the respondent some time to think about it, and to tell you their answer (especially for open questions). If you sense that they might not understand what you’ve asked, don’t rephrase the question immediately. Instead, ask if they need any clarification about what was asked.

If you don’t understand something a respondent has told you (or maybe you missed something while taking notes), feel free to ask for clarification – but let them finish their initial thoughts first. This is particularly helpful when our community partner uses jargon that is common to them but unfamiliar to us.

Most importantly, especially for open questions, actively listen to what the respondent is saying. There will be times when you might wonder why the respondent is telling you a long and winding story that, in your mind might not even seem relevant to the question that was asked. However, allowing them to tell their story builds trust. Additionally, information that might be revealed in their stories will become extremely useful later on during development.

For example, while listening to responses from one of our community partners, we learned that many of the users of the tool we were developing for Community Living Cambridge enjoyed video games. While this wasn’t within the scope of the initial challenge, many students in CIS3750 opted to integrate that knowledge into their solution.

Take care of yourself as well. Since our community partners are often working with people who are marginalized, traumatized, abused, or misunderstood by society or its systems and structures, you may hear or see things that are shocking or painful, or that negatively affect your mental health. If you feel uncomfortable, contact the teaching team or use the resources listed in the course outline. Our classroom work is important, but your health is more important.

Finally, thank your respondents for offering you their experience, knowledge, and time.

Activity 1: What’s Wrong With These Questions?

Review Things To Avoid, then identify any issues you see with each of the following questions. How would you rewrite these questions so that they are good questions?

• On a scale of 1 to 4, where 1 is “satisfied” and 4 is “extremely satisfied”, please rank your response to the question “How satisfied were you with your service today?”
• How bad is the Federal Carbon Tax, and how do you think it will impact federal employees at the CRA?
• Should concerned parents vaccinate their children?
• How satisfied are you with CIS3750 and your overall experience in the School of Computer Science?
• How amazing is our new and improved double-decker chocolate raspberry-filled donut?
• When you eat donuts, do you prefer them warmed up?
• Are there any perceived benefits to expanding STEM to the more inclusive STEAM?
• Mitacs provides federal and provincial matching dollars to researchers working with SMEs and other industry partners to offer 4 to 8-month-long graduate student internships. Do you think taxpayers should be burdened with the cost of this program?

Review Questions