According to several studies, the larger the software project, the less chance of success it will have because the complexity exceeds the developer’ ability to comprehend it. The key for success depends on the ability of the system architect to decompose requirements into smaller tasks so that developers can understand and be able to implement them. When developers finished, these tasks can be integrated into the large system according to the defined architecture.

A good architecture must detail all interfaces, functions, and control mechanisms separately so it can be flexible, easy to change. If done correctly, it allows developers to concentrate on specific task without the need to focus on interfaces, data flow, and other system functions. If documented well, it reduces the need for coordination between different team functions (Developers, testers, quality assurance, configuration management) but if done poorly, it will be a never-ending arguments, confusions, and lots of integration problems.

Architecture flows from the requirements to functional specification. Therefore, requirements must be correct, complete, and unambiguous. A poorly defined requirements will lead to bad architecture and bad architecture will increase problems in large projects. Obtaining requirements is the key responsibility of the system architect. He must work with the customers representative such as the Business manager to identify the needs and the system’s goals. The system architect is the interface between users and developers because users do not know how to explain their needs in the way that developers understand, and developers do not know about business process to ask the right questions. To reduce the risk in large project, the system architect must build prototypes to validate the requirements with customers. Prototyping is the process of building a “model of the system” because it converts the intangible requirements into a tangible but limited working model of the desired information system.The system architect communicates to developers by specifying “black-box” descriptions of the tasks. Black boxes are abstract entities that can be understood, and implemented independently from the rest of the system. The process of building black-box models is called “abstraction”. Abstraction is used to simplify the design of a complex system by reducing the number of details that must be considered. Basically, the architect explain to developers what to implement, but not how to implement it.

Developers must focus on building software that are dependable and free from defects. To minimize number of defects, they must follow the project’s defined process using best practices and tools that can help them to find defects early during reviews and tests. Developers also need to measure their works to ensure software quality. By having measurements, developers are having useful feedback on their works and provides reassurance to the system architect, project manager and users that the software is of high quality.

Developing large project with quality is not easy. It requires strong discipline, teamwork, and lot of coordination. It also require a strong management supports in understanding the hard works and giving developers more time to design and test their works. Most large projects are very complex so the key factor is a good architecture and strong project management. These skills are hard to find because the training today is still focusing on the programming aspect NOT the designing aspect and management aspect. Very few schools teach architecture, design and project management but only programming languages and that is why many companies are successful with small projects but NOT large projects. Software and system architecture are difficult to teach as it requires a lot of experiences in real works. You can not learn it from books or take a class but must experiencing it in real projects, especially large projects. The best way to learn about architecture is to be an assistant to the system architect, this “On the job training” would be good to developers that have several years of experiences. Once you experiencing it and have some basic knowledge about it then you can take few classes to strengthen your knowledge and learn more about different styles, different approaches. Only when you have mastered these techniques then you could apply for the system architect position. Remember that developers build their works upon the architecture and it is the architecture that determines whether the project will be successful or not.

Architecture & design phase

A student asked: “What is the difference between an architecture phase and a design phase? I am confused because the software lifecycle only mentions requirements, design, code, and test. What happen in the architecture phase? What does an architect do in this phase? Please explain.”

Answer: Basically, architecture phase is the highest level of abstraction of a software system and the design phase is the lower level that focuses on the details such as modules and components. Architecture is the allocation of system requirements (Hardware, Software, Interface etc.) to system components as well as the interaction between these components. Design usually deals with the functioning of each component in more details. For example, when you build a house you need an architect to design all the components of a house such as foundation, columns, roofs and walls but you do not need the architect to design the detailed kitchen or a bed rooms in the house. These details are often given to construction workers and the interior decorators.

When you build a large and complex software system, you need to divide the system into smaller components for easier implementation. Architecture phase is an essential step to make sure all these components are defined and the interaction between them is clearly identified. Sometime people consider architecture phase as the “high design” phase and design as the “detailed design” phase. The waterfall lifecycle considers both phases as “design” to keep it simple and easier for students to learn. Since most projects in school are small and simple where hardware and the interaction with software are known, students do not concern with hardware can start with the design phase of software. However in the industry, most software projects are large and complex and you need to know all components of the entire system (Both hardware and software) so you must learn software architecture well in order to build a quality system.

In the architecture phase, you must identify the context and scope of the project by set up boundary based on the customers’ requirements. You need to define both functional as well as non-functional views and all constraints. You need to identify all components such as the process view, logical view, physical view, interface view, infrastructure view, operational view, and security view for a complete software system.

Most software developers often pay attention to the functions of the system but not pay enough attention to the non-functional views (The quality attributes). Because most non-functional views are not well defined, projects often get into problem later because these are what customers’ expect but rarely mention in the requirements. This is where experienced developers are doing better than inexperienced ones as they learn through years of working by knowing what to ask customers. The customers may say: “The system must be fast”, it is a non-function requirements but that is too vague. Experienced developers know how to ask customers for more specific details to make sure that this request is measurable, and testable. (How fast do you want? What do you mean by fast? How many second? Etc.) They know that to achieve customers’ satisfaction they need to make sure that all of the important non-functional views are defined. These include the runtime characteristics such performance, scalability, availability and security etc. That is why the role of requirements engineers is so important in larger and complex system to clearly define these non-functional requirements.

Because most of the non-functional requirements are systemic issues, the architecture phase must identifies them as well as defines the interaction among them just like when building a house the architect must calculate the strength of the foundation and all the columns that keep the house in place. For example, larger house requires better foundation and more columns to withstand the weight of the house. For this reason the architecture phase is important in large software system. When all the functional and non-functional requirements are defined, the next step is to focus on how each component is implemented. In this phase, the software architect decides on what technologies are used, which components are built in house, which ones are brought from vendors, which one can be outsourced. They have to make decision based on factors such as cost, licensing, vendor relationships, compatibility, interoperability, support, and user environments etc. These decisions are vital to the success of the project because they are risks that must be deal with. Software architect must reducing risk where there is high degree of complexity or uncertainty by conducting trade-off analysis to manage risks and make sure the project can be done within costs and schedules. All decisions must be reviewed and evaluated by the architect, team leaders, project manager and customers.

The main problem with large and complex software system is that it is difficult for people to understand how these components work together. That is why software architect must use tool such as the Unified Modeling Language (UML) diagrams to convey the architecture to others. The architect review is where the project team, the support team (SQA and SCM, database specialists, security specialists etc.) as well as management and customers must participate to understand how the entire system will work. The software architect must make sure that the defined architecture is understood by everybody involved. Of course, development team must understand them since they will have to design and implement each component but support team must also know the architecture well. Configuration Management people must set a baseline to assure that all changes are under controlled. Quality Assurance must review all architecture components and make sure that they comply with standards and procedures. Security specialist must make sure that the architecture has taken necessary steps to ensure the integrity of the system etc.

If the architecture is approved, the next step is assign team members to start the design process for each component. Although the architect does not involved in the design and coding tasks, but he must actively helping team members to make sure that they are doing it correctly within the defined architecture. Basically the architecture phase is the most important in large software project. If it is done well, there will be les problems later but if it is not well defined, the chance that the project will fail may be very high. A poorly defined architecture will lead to bad design and bad design will lead to project failure and that is why so many software projects fail. They never fail because of coding but because of bad design and poorly defined architecture. That is why I think software engineering that covers all phases of software development is needed for all software students.