4.4 Data Gathering in Software Development II

 

Data gathering is a vital process in software development, involving systematic interaction between software engineers (often acting as business analysts) and customers, including users and stakeholders. The objective is to collect comprehensive information that shapes the requirements of the software system being developed. Understanding the various techniques, their advantages, and their limitations is essential for ensuring that the final product aligns with user needs and business goals.

Techniques for Data Gathering

There are several primary techniques used for gathering data, each with distinct strengths and weaknesses. These techniques include:

1. Interviews
2. Group Meetings
3. Observation
4. Temporary Job Assignments
5. Questionnaires
6. Document Review
7. Software Review

Each technique serves unique purposes and can be more or less effective depending on the context and the specific information needed.

Table 1: Summary of Data Collection Techniques

Technique	Strengths	Weaknesses
Interviews	- Gather qualitative and quantitative data	- Requires skill to conduct effectively
	- Obtain detailed insights	- Potential for biased results
	- Good for surfacing requirements	- May yield misleading or irrelevant information
	- Requires triangulation to verify results	- Not practical for large groups (e.g., over 50 people)
Group Meetings	- Facilitate decision-making	- Can be time-consuming with many participants
	- Collect detailed and summarized information	- Interruptions can divert attention
	- Engage multiple users	- Political arguments may arise
	- Wrong participants can lead to poor outcomes
Observation	- Uncover unarticulated procedures and reasoning	- May not represent typical behavior
	- Unbiased by opinions	- Observer presence can alter behavior
	- Provides a deep understanding of the problem domain	- Time-consuming
Temporary Assignments	- Gain insights into current context and challenges	- May not represent typical activities
	- Basis for further questioning	- Time-intensive
Questionnaires	- Anonymity encourages honest responses	- Recall may be imperfect
	- Efficient for large-scale data collection	- Unanswered questions limit information
	- Good for cultural assessments	- Questions may be misinterpreted
Document Review	- Understand organizational history and context	- May introduce bias into future design
	- Saves time by reducing user interviews	- Not effective for capturing attitudes
Software Review	- Learn about current work procedures	- May not reflect current practices
	- Identify questions for users	- Can be outdated or inaccurate

Meetings

Meetings are gatherings of three or more individuals for a fixed duration, aimed at discussing a small number of topics and reaching consensus decisions. They can serve as a complement to interviews or even replace them by providing a forum for collective discussions among users to work out requirements and alternatives for an application.

Advantages of Meetings

• Surface Requirements: Meetings can be instrumental in surfacing and clarifying requirements.
• Reach Consensus: They foster group decision-making and consensus building.
• Gather Information: Both detailed and summary information can be obtained from discussions.

Disadvantages of Meetings

Despite their benefits, meetings can also be a significant waste of time, especially if not well-structured. Generally, larger meetings yield fewer decisions and take longer. Therefore, it's essential to have a well-prepared meeting plan that includes:

• Defined Agenda: Circulate the agenda in advance, restricting the number of topics to between one and five.
• Fixed Duration: Limit meetings to no more than two hours to maintain participant engagement.
• Decision Checkpoints: Incorporate specific checkpoints for required decisions throughout the meeting.
• Minutes and Follow-Up: Document discussions and decisions, identifying responsible persons and deadlines for any follow-up items.

Joint Application Development (JAD)

Joint Application Development (JAD) is a specialized form of meeting where users and technicians collaborate continuously over multiple days to identify application requirements. Before a JAD session, users are trained in requirement documentation techniques, particularly in diagramming data and processes. They also prepare by documenting their own jobs, collecting relevant materials such as forms, reports, and inputs.

JAD Session Structure

• Duration: JAD sessions typically last between 3 to 8 days, with sessions ranging from 7 to 10 hours each day.
• Objective: The goal is to gather all interested parties in one place to define application requirements efficiently.
• Outcome: Studies show that JAD can compress the analysis phase from three months into about three weeks, while still producing comparable results.

However, a potential drawback is the risk of user interruptions, which can divert attention and hinder the objectives of the JAD sessions.

Observation

Observation involves manually or automatically monitoring individuals' work processes. In manual observation, an analyst sits with individuals, taking detailed notes of their activities and steps during work. Automated observation utilizes software to track usage patterns, communications, and actions performed on a computer.

Advantages of Observation

• Unarticulated Insights: It is particularly useful for capturing information from users who may struggle to articulate their processes.
• Protocol Documentation: Observers can create detailed protocols that outline minute-by-minute actions, providing insights into reasoning and decision-making.
• Objective Understanding: Observers gain a firsthand understanding of current work environments without the influence of personal opinions.

Disadvantages of Observation

• Non-Representative Timing: The observed period may not accurately represent typical activities.
• Behavior Changes: The presence of an observer can alter the behavior of the individual being observed.
• Time-Consuming: This method may require significant time investment without necessarily yielding better insights than other methods.

Ground Rules for Observation

To ensure ethical and effective observation, establish clear ground rules:

• Define Scope: Identify specific activities to be observed and the duration of observation.
• Obtain Approval: Secure both management and participant consent before beginning.
• Transparent Purpose: Clearly explain to participants how the information will be used and why it is being collected.

Temporary Job Assignment

Temporary job assignments allow individuals to immerse themselves in the roles they are studying. This hands-on experience provides a deeper understanding of the tasks involved and the complexity of the work.

Advantages of Temporary Assignments

• Firsthand Experience: Assignees gain a comprehensive appreciation of tasks and contextual terminology.
• Informed Questioning: Knowledge acquired helps formulate relevant questions about existing processes.

Disadvantages of Temporary Assignments

• Time-Consuming: These assignments can take considerable time and may not always yield representative experiences.
• Potential Bias: The assignee may develop biases that could affect future design decisions.

Questionnaires

Questionnaires are structured forms designed to gather information from a large audience. They can be used in paper or digital formats, and they offer the advantage of anonymity, often leading to more honest responses than interviews.

Types of Questions

• Open-Ended Questions: These allow for expansive responses and are useful for gathering qualitative data. An example is: "What features do you believe should be included in the new application?"
• Closed-Ended Questions: These require specific answers, such as yes/no or multiple-choice. For example: "Do you use the current reporting tool?"

Reliability and Validity

• Reliability: Refers to the consistency of the questionnaire results. A reliable questionnaire produces the same answers over repeated trials.
• Validity: Ensures that the questionnaire measures what it intends to measure. It is crucial to avoid subjective questions that could lead to inaccurate self-assessments.

Guidelines for Developing Questionnaires

Step	Description
1. Define Goals	Determine the specific information required and identify the target respondents.
2. Select Question Types	Choose open-ended questions for qualitative insights and closed-ended questions for factual data.
3. Group Questions	Organize questions by topic or type for clarity and coherence.
4. Review for Clarity	Check for ambiguous questions and ensure comprehensiveness of response options.
5. Pilot Test	Test the questionnaire with a small group to identify issues and gather feedback.
6. Analyze Feedback	Revise questions based on feedback to enhance clarity and effectiveness.
7. Finalize and Distribute	Prepare the final version and provide clear instructions for completion and return.

Document Review

Document review involves studying existing documentation related to policies, procedures, and systems. This can provide insight into current workflows and organizational practices.

Types of Relevant Documents

• Internal Documents: Policies, procedures, user manuals, organization charts, performance standards, training documents, etc.
• External Documents: Technical publications, industry reports, regulatory information, and market research.

Advantages of Document Review

• Objective Insights: Documentation offers factual information devoid of personal biases.
• Background Knowledge: It helps analysts understand existing workflows and identify potential issues.

Disadvantages of Document Review

• Limited Insight into Attitudes: Documents may not capture user perceptions or motivations, which are often crucial for understanding work processes.
• Potential for Outdated Information: Some documents may not reflect current practices.

Software Review

Software review entails examining existing applications to understand current work processes and how they are influenced by software design. This analysis can inform discussions about user needs and constraints.

Advantages of Software Review

• Current Workflows: Provides insights into how tasks are performed and the extent to which software constraints impact efficiency.
• Identifying Gaps: Helps in understanding what features users would like to retain or discard in new applications.

Disadvantages of Software Review

• Documentation Issues: Existing software may lack accurate or current documentation.
• Code Readability: The underlying code may be poorly documented, making it challenging to extract useful information.

Conclusion

Collecting information for software applications is a multifaceted process that encompasses various techniques, including interviews, group meetings, observation, questionnaires, document reviews, and software reviews. Each method has its unique advantages and disadvantages, and selecting the appropriate technique depends on the specific context and information needs of the project. Understanding these methods' nuances ensures that software engineers can effectively gather the necessary data to inform development and meet user requirements.