Software Engineering: Foundations, Processes, Requirements, Design, Testing, and Maintenance

Software engineering is the disciplined development, operation, and maintenance of software systems using systematic methods, tools, and quality controls. The field emerged as a response to the software crisis identified in the late 1960s, when increasingly large systems were delivered late, over budget, difficult to maintain, or not delivered at all.2 The NATO conferences of 1968 and 1969 helped popularize the term and positioned software creation as an engineering activity rather than only a programming craft.

A useful historical distinction is between a program and a programming system product: a single program becomes far more complex when it must be generalized for multiple users, fully documented, tested, integrated, packaged, and maintained as a real product. This helps explain why software projects scale nonlinearly in cost and coordination effort.

Software also has unusual characteristics compared with physical products. It is largely intangible, does not wear out in the mechanical sense, is often highly changeable, and accumulates complexity through modification over time.2 These properties make process discipline, requirements clarity, design structure, and testing rigor essential.

Fred Brooks' famous argument in No Silver Bullet remains foundational: there is no single technology or management technique likely to produce a tenfold improvement in productivity, reliability, or simplicity within a decade. Brooks separated essential complexity from accidental complexity, arguing that many advances reduce the accidental part, but the core problem remains difficult. This perspective still explains why modern tools help greatly, yet do not eliminate the need for sound engineering judgment.

Software development is therefore studied as an integrated lifecycle involving requirements, design, implementation, testing, deployment, maintenance, measurement, and evolution.2

Footnotes

1. History of software engineering - Historical overview of software engineering, software crisis, structured methods, object orientation, and agile evolution. ↩ ↩² ↩³ ↩⁴ ↩⁵

2. The history of coding and software engineering - Describes the software crisis, NATO conferences, and subsequent development of software engineering practices. ↩

3. No Silver Bullet - Summary of Fred Brooks' argument about essential and accidental complexity in software engineering. ↩ ↩² ↩³ ↩⁴ ↩⁵

4. Software Testing: Boundary Value Analysis & Equivalence Partitioning - Explains boundary-focused testing and the role of regression-like retesting concepts in ongoing software change contexts. ↩

Module I — Introduction and Software Development Life Cycles

A software process defines how software is specified, designed, built, validated, and evolved. Different lifecycle models attempt to control uncertainty, improve visibility, and reduce risk. The right model depends on requirement stability, technical novelty, stakeholder access, schedule pressure, and risk tolerance.

Core introductory concepts

Characteristics of software

• Intangible and logic-based rather than manufactured material goods.
• Easy to copy, but expensive to design, validate, and evolve.
• Does not physically wear out, yet degrades structurally through poor changes and increasing complexity.
• Often custom-built and continuously modified, which makes maintenance a dominant lifecycle cost.2

Software myths Software myths are false beliefs held by customers, managers, or developers, such as: requirements can change freely with little cost, testing alone guarantees quality, or programming is the only meaningful work in a project. These myths lead to underestimation, poor planning, and weak engineering discipline.

Major SDLC models

A critical comparison shows that no lifecycle model is universally superior. Waterfall emphasizes control, incremental and evolutionary models emphasize adaptability, and the spiral model emphasizes risk analysis. Reuse-oriented development changes the lifecycle itself because requirements and design may be constrained by available components.

Non-traditional software development processes

Rational Unified Process (RUP). RUP is an iterative framework organized into inception, elaboration, construction, and transition phases, combining use cases, architecture focus, and iterative risk reduction.

Rapid Application Development (RAD). RAD prioritizes fast delivery through prototyping, component reuse, user involvement, and time-boxing. It works best when scope is partitionable and speed is critical.

Agile development. Agile methods emphasize short iterations, stakeholder feedback, adaptive planning, and working software over excessive upfront specification. Agile does not eliminate engineering discipline; it relocates it into continuous collaboration, automated validation, and iterative refinement.

Footnotes

1. History of software engineering - Historical overview of software engineering, software crisis, structured methods, object orientation, and agile evolution. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷ ↩¹⁸ ↩¹⁹

2. Software Testing: Boundary Value Analysis & Equivalence Partitioning - Explains boundary-focused testing and the role of regression-like retesting concepts in ongoing software change contexts. ↩

3. Ch 4: Requirements Engineering - Covers requirements engineering processes, SRS structure, validation, and distinctions among requirement types. ↩

Module II — Requirements Engineering

Requirements engineering is the foundation of successful software projects because errors introduced here propagate into design, code, testing, and maintenance at much higher correction cost later. A requirement is a condition or capability needed by a user or system to solve a problem or achieve an objective.

User needs, software features, and software requirements

A user need describes what stakeholders are trying to accomplish. A software feature translates that need into an externally meaningful capability. A software requirement expresses the capability or constraint in a form suitable for analysis, validation, and contractual agreement.

Classes of user requirements

A common distinction separates:

• Enduring requirements: relatively stable needs tied to core business or domain operations.
• Volatile requirements: needs likely to change because of policy, market, regulation, technology, or organizational change.

Volatile requirements demand mechanisms for prioritization, traceability, negotiation, and controlled change management.

Functional and nonfunctional requirements

Functional requirements describe what the system shall do, such as calculate totals, authenticate users, or generate reports. Nonfunctional requirements describe qualities and constraints such as performance, availability, safety, usability, portability, reliability, and security.2

Examples:

Sub-phases of requirement analysis

Requirement work usually includes elicitation, analysis, specification, validation, and management. Elicitation gathers information from interviews, observation, workshops, documents, and prototypes. Analysis resolves ambiguity, inconsistency, overlap, and feasibility issues. Specification records requirements clearly. Validation checks correctness, completeness, consistency, realism, and verifiability. Management tracks changes over time.

Barriers to eliciting user requirements

Requirement elicitation often fails because stakeholders use different vocabularies, omit tacit knowledge, disagree on priorities, or cannot envision future workflows clearly.2 Additional barriers include incomplete domain knowledge, geographically distributed teams, political conflict, ambiguous language, and changing business context.

Software requirements document and SRS standards

The Software Requirements Specification (SRS) is the official statement of what will be implemented, including user requirements, detailed system requirements, and relevant constraints; importantly, it is not a design document. IEEE Std 830 historically standardized SRS structure, and later ISO/IEC/IEEE 29148 modernized requirements engineering terminology and specification guidance.2

A good SRS should be:

• Correct
• Unambiguous
• Complete
• Consistent
• Ranked for importance or stability
• Verifiable
• Modifiable
• Traceable2

Case study idea: real-time system SRS

For a real-time monitoring system, the SRS must usually combine functional behavior with strict timing, event handling, fault tolerance, and external interface constraints. Real-time SRS documents must specify response deadlines, sensor data rates, alarm logic, fail-safe behavior, and recovery conditions in measurable terms.

Footnotes

1. Ch 4: Requirements Engineering - Covers requirements engineering processes, SRS structure, validation, and distinctions among requirement types. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹² ↩¹³ ↩¹⁴ ↩¹⁵ ↩¹⁶ ↩¹⁷

2. What Are Nonfunctional Requirements? Definition + Best Practices - Explains nonfunctional requirements and their role in product development and SRS documentation. ↩ ↩²

3. Overcoming Challenges of Requirements Elicitation in Offshore Outsourced Software Development Projects - Discusses common elicitation barriers such as ambiguity, incompleteness, inconsistency, and stakeholder challenges. ↩ ↩²

4. Markdown Software Requirements Specification (MSRS) - Practical SRS template aligned with IEEE 830 and ISO/IEC/IEEE 29148, emphasizing traceability and verifiable requirements. ↩ ↩²

Module III — Software Design, Metrics, and Development

Software design determines how the system will satisfy requirements through decomposition, interfaces, control, data organization, and component interaction. Good design aims for correctness, maintainability, adaptability, understandability, reusability, and testability.2

Design strategies and methodologies

Design may proceed top-down from system goals to modules, bottom-up from reusable components to larger assemblies, or through hybrid methods. Data-oriented design emphasizes how information is organized, stored, and transformed across the system.

Structured design

Structured design uses decomposition and clear control hierarchy to organize modules. A structure chart shows calling relationships and module breakdown. Two major evaluation principles are:

• Cohesion: higher cohesion is desirable because each module serves a focused purpose.
• Coupling: lower coupling is desirable because modules interact through limited, stable interfaces.

A modular design with strong packaging improves maintainability, testing, and incremental development.

Object-oriented design

Object-oriented design organizes software around objects, classes, responsibilities, and collaborations. It benefits from abstraction, encapsulation, inheritance, and polymorphism. Design patterns capture proven arrangements such as Factory, Observer, Strategy, and Singleton for recurring software problems.

Structured analysis

Data Flow Diagram (DFD) shows processes, data stores, external entities, and data movement, helping analysts understand functional decomposition. A data dictionary defines data items, structures, meanings, ranges, and usage rules.

Software measurement and metrics

Metrics help estimate size, complexity, quality, and effort. Common size-oriented measures include Lines of Code, Function Point counts, and other effort indicators. Function points are useful because they focus on externally visible functionality rather than implementation language alone.

Halstead metrics estimate software attributes from the number of distinct and total operators and operands in code. These metrics attempt to characterize vocabulary, length, volume, difficulty, and effort.

Cyclomatic complexity measures structural complexity from a control flow graph and is closely tied to testing effort. For a single connected component:

$$V(G) = E - N + 2$$

where $E$ is the number of edges and $N$ is the number of nodes in the control flow graph.

It may also be expressed as:

$$V(G) = P + 1$$

where $P$ is the number of predicate nodes.

Higher values generally indicate more branching and more independent paths requiring test attention, though the metric must be interpreted with design context rather than used mechanically.

Development practices

Language selection should consider ecosystem, performance, safety, team skill, tooling, maintainability, and deployment constraints. Coding guidelines improve readability, consistency, defect prevention, and review quality. Code documentation should capture interfaces, assumptions, side effects, invariants, and usage rather than merely restate the code.

Footnotes

1. History of software engineering - Historical overview of software engineering, software crisis, structured methods, object orientation, and agile evolution. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

2. A testing methodology using the cyclomatic complexity metric - NIST publication explaining control flow graphs, cyclomatic complexity, and links to structured design principles. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹ ↩¹²

Module IV — Software Testing

Software testing checks whether implemented software satisfies specified behavior and quality expectations. Testing is not only bug hunting; it is also a validation activity linked to requirements, design structure, risk, and operational confidence.

Testing process

A typical testing process includes planning, test design, test case preparation, environment setup, execution, defect logging, regression testing, reporting, and closure. Good test design seeks representative and high-risk scenarios rather than exhaustive execution, which is usually impossible.

Functional testing

Functional testing derives tests from specifications and business rules rather than code structure.

Important techniques include:

• Boundary value analysis: choose values at minimum, maximum, just inside, and just outside valid boundaries because defects frequently occur at edges.
• Equivalence class testing: divide the input domain into valid and invalid classes and test representative members to reduce redundancy.
• Decision table testing: effective for systems with complex business combinations.
• Cause-effect graphing: helps derive tests for logical combinations of inputs and outputs.

Structural testing

Structural testing uses program logic to derive tests. This includes path testing, data flow testing, and mutation testing.

• Path testing targets independent execution paths, often guided by cyclomatic complexity.
• Data flow testing focuses on definition-use chains of variables.
• Mutation testing introduces small changes to code and checks whether test suites detect them, giving insight into test effectiveness.

Levels of testing

• Unit testing: tests individual functions, methods, or classes.
• Integration testing: tests interactions among modules.
• System testing: tests the complete integrated system against requirements.
• Alpha testing: in-house acceptance-like testing in a controlled environment.
• Beta testing: limited external release to real users for practical feedback.

Debugging and standards

Debugging is the diagnosis and correction of defect causes after test failure detection. Testing tools support test management, automation, static analysis, coverage, and defect tracking. Standards and disciplined processes improve repeatability, traceability, and evidence of quality in both general and regulated development settings.

Footnotes

1. Black Box Testing Techniques Explained - Overview of functional testing methods including equivalence partitioning, boundary value analysis, decision tables, and cause-effect graphing. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰ ↩¹¹

2. A testing methodology using the cyclomatic complexity metric - NIST publication explaining control flow graphs, cyclomatic complexity, and links to structured design principles. ↩

Module V — Software Maintenance and Evolution

Software maintenance includes correcting faults, adapting software to changed environments, improving performance, and enhancing functionality after release. In practice, maintenance consumes a substantial share of total lifecycle effort because software continues to evolve with user needs, platforms, regulations, and operational experience.2

Management of maintenance

Maintenance must be planned, staffed, documented, prioritized, and controlled. Requests should be classified by urgency, impact, risk, and business value. Good maintenance management relies on issue tracking, versioning, traceability, release planning, and regression testing.

Maintenance process and models

A maintenance process usually includes change request logging, impact analysis, modification, retesting, documentation update, configuration update, and release. Maintenance models vary by organizational context, but all must preserve system integrity while accommodating change.

Regression testing

Regression testing is crucial in maintenance because even small fixes can break previously working behavior.2 Automated regression suites are especially valuable when systems evolve frequently.

Reverse engineering and reengineering

Reverse engineering extracts higher-level understanding from existing code, data, or documentation. Software reengineering restructures or modernizes systems while preserving useful functionality. These are essential when organizations depend on legacy systems with poor documentation or outdated technology.

Configuration management and documentation

Configuration management ensures that the correct versions of code, documents, tests, and builds are identified and controlled. Without it, maintenance becomes error-prone and auditability is lost. Updated documentation is equally important because undocumented changes increase future maintenance cost and risk.

Footnotes

1. Software Testing: Boundary Value Analysis & Equivalence Partitioning - Explains boundary-focused testing and the role of regression-like retesting concepts in ongoing software change contexts. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹

2. History of software engineering - Historical overview of software engineering, software crisis, structured methods, object orientation, and agile evolution. ↩

3. Black Box Testing Techniques Explained - Overview of functional testing methods including equivalence partitioning, boundary value analysis, decision tables, and cause-effect graphing. ↩

Integrative View Across All Modules

Software engineering should be studied as a connected discipline rather than isolated topics. Weak requirements degrade design; poor design increases coding and testing difficulty; inadequate testing increases maintenance cost; poor maintenance discipline erodes long-term quality.5

A coherent view of the subject can be summarized as follows:

1. Introduction and process establish why disciplined methods are necessary.3
2. Requirements engineering determines what problem is actually being solved.3
3. Design and metrics organize the solution into manageable structures and measurable artifacts.
4. Testing provides evidence that the implementation satisfies expectations and handles failure cases.
5. Maintenance and configuration management preserve usefulness after delivery.

This integrated perspective explains why software engineering is not synonymous with programming alone. It is an end-to-end engineering discipline concerned with quality, process, structure, validation, and evolution.

Footnotes

1. History of software engineering - Historical overview of software engineering, software crisis, structured methods, object orientation, and agile evolution. ↩ ↩² ↩³

2. Software Testing: Boundary Value Analysis & Equivalence Partitioning - Explains boundary-focused testing and the role of regression-like retesting concepts in ongoing software change contexts. ↩ ↩²

3. Ch 4: Requirements Engineering - Covers requirements engineering processes, SRS structure, validation, and distinctions among requirement types. ↩ ↩²

4. A testing methodology using the cyclomatic complexity metric - NIST publication explaining control flow graphs, cyclomatic complexity, and links to structured design principles. ↩ ↩²

5. Black Box Testing Techniques Explained - Overview of functional testing methods including equivalence partitioning, boundary value analysis, decision tables, and cause-effect graphing. ↩ ↩²

6. No Silver Bullet - Summary of Fred Brooks' argument about essential and accidental complexity in software engineering. ↩

7. The history of coding and software engineering - Describes the software crisis, NATO conferences, and subsequent development of software engineering practices. ↩

8. Overcoming Challenges of Requirements Elicitation in Offshore Outsourced Software Development Projects - Discusses common elicitation barriers such as ambiguity, incompleteness, inconsistency, and stakeholder challenges. ↩

9. Markdown Software Requirements Specification (MSRS) - Practical SRS template aligned with IEEE 830 and ISO/IEC/IEEE 29148, emphasizing traceability and verifiable requirements. ↩