Artikel

 

[IT Project Planning / Software Engineering / Application Design & Architecture]

 

Software Architecture means having the 'big picture' in front of your eyes, evaluating a project from different standpoints. This requires knowledge in many different fields. I define the role of a software architect as the one of a "hands-on cross-border IT specialist", who verifies technical feasibility and options as early as possible, mostly in the first meeting when the initial ideas about the business purpose of the project are brought to the table.

GENERAL CONCEPT PHASES
I want to present the general planning phases - well knowing that they interact, overlap, and spark sub-concepts and detail-concepts:

• Feasibility concept: (general verification and discussion of the initial project idea)
• Business rule (purpose) concept: defining 'Use Cases', 'Goals', user interfaces, etc. (supported through Logical or abstract data models)
• Technical concept: detailed technical specification, physical data model, ... a practical guide for developers

Constant verification of the concepts is essential during the planning phase - otherwise one ends up with starting the implementation and discovering (after having spent probably hundreds of hours) that one ran into an issue that can question the entire concept! Continuous and competent communication between all participating organizational units is key to the success of a project. Full commitment of all participating players and full back-up by the involved management levels and stakeholders is important in order to finish the project successfully and without hassle in between. Resource management and assessment has to be done early to avoid ending up in understaffed projects that cannot flow smoothly because of too much pressure or a lack of competence or determination by the leaders/team or altogether.

THE GENERAL PROCESS
How to design a software application?
The theoretical process in brief:

• Gathering information from all involved parties (internal / external)
• Identifying logical, participating entities and defining their relations
• Defining process flow, participating modules, communication types, phases/milestones
• Identifying critical/unclear task units, preparing a question catalog
• Definition how logical entities will be translated into physical models
• Definition of the type of data storage; db design and/or XML schemata
• Review, Verification
• prototyping / GUI design
• Review, Verification
• Building the application: Top-Down, Down-Top, or a mix of both realization strategies
• Documentation, Test, Change-/Implementation plan for production environment

Please note that especially the planning tasks may overlap or that the order may change slightly.


The information gathering phase includes not only to find out what the business purpose of the application system should be and which processes have to be covered and automated but also to recognize future scalability potential, limitations, inconsistencies, and so on.
From what a software engineer / application architect hears and gets as information he/she has to fully understand what the client (internal department or external) wants (... I call it recognizing the vision.) and then build his/her question catalog which covers implications, inconsistencies, verifications, scalability, etc. regarding the project.

A task of the software engineer is to clarify in how far issues are covered that are not necessarily top ranking issues from the business standpoint, among these points are for example disability support, security, different platform / web client (e.g. browsers) support. All questions will help to clarify the scope of the project.

The entire planning (thinking) phase (the time before anything is realized) is the most important part of the design process. Before physically building an application I assemble all parts brain-wise, let the application run there.
Playing through all possibilities and checking how the application would theoretically react on all kinds of input situations, malicious access attempts, error situations, etc. is essential.
Don't start coding too early - not before the entire application including all task units work seamless together in your 'brain'.
Especially don't start with the 'easy stuff' but instead start coding and verifying 'unclear', difficult units and look what the results are, if they work out as the concept envisions - only if you can be sure that a certain technique will work in practice, you should go ahead with it as part of your application design.

Going theoretically through your application means sitting and staring holes into the air - not productive? Oh yes, it is.
It guarantees that you will go through the actual application building process without / or at least with less unexpected situations.
Application design also means to adapt the grade of scalability and find/suggest the way to go: neither plan too big, nor too small. Expect growth of the data volume and the organization and foresee possible wishes. Plan phase I but have phase II in mind.

The entire process is a back and forth between the participating project players and will eventually result in modified specifications, some compromises but most important: a solid concept and roadmap for the project which defines milestones, duties, resources, data, input, modules, just everything you came across during the planning phase.

The grade of formality of such concepts depends on the company. UML tools are quite common ('and one can either love or hate them') but other methods to 'get the act together' are perfectly fine as well.

+++ CONSIDERATIONS ALONG THE WAY

Standards
An investment in the future is definitely the development of pamphlets that describe 'programming standards' and 'naming standards' which have to be followed by the staff. The creation of standards takes time and involves costs, but the maintainability of all systems based on these standards will profit of such efforts.

Prototyping
A prototype is an example where the application is demonstrated without having the real functionality under the hood. It helps all participating business units and yourself to be sure that this is what everybody wants. And it is perfect to simulate the work flow and to review, improve the usability. It's a chance to walk in the shoes of the future audience...

Methodology of Coding
Top-Down development vs. Down-Top:
Going from top to down means that you start building top functionality first, having in the beginning a very basic application running, mostly without details but which shows a work (page, screen, process) flow. Then implement piece by piece the detailed functionality.

The reverse approach is to start building a certain detailed functionality of the application first, then add another, and so on. The overall application grows from time to time and shows their complete face later.

Often it will be a combination of these approaches.

DB Design
Database design as part of the application design process is an integral part.
Not overseeing the 'big picture' of the application may lead to a design that is of course theoretically perfectly normalized but it might produce an overhead (and performance loss) when data is retrieved or inserted.

This is by far no statement against normalization. Normalization is the guarantor for easier maintainable applications.
Anyhow, that doesn't mean that there's only one single possible db design outcome. Many ways lead to Rome.... How to find the right design depends on different aspects.
Scalability is e.g. a factor that might play a role. Don't plan for unlimited scalability unless the nature or the expected lifetime of the product requires it. That might lead to a simpler data model.
I have seen catastrophic database designs that made it an odyssey to retrieve data - 'millions' of Joins, temporary tables, and much other stuff that scares the hell out of performance geeks slowed down the performance even of trivial queries. A slight shifting of information and reorganization (still having data in a normalized manner!) can eliminate such bottlenecks. Database designer should have in mind how data will be retrieved later. Don't live in an isolated theoretical world.

Design a database (it's tables and relationships, it's keys, indices, etc.) by the rules. But there's not only one way to do it. Think about performance which involves knowing about data volume and types of access. How will indices, keys and inserts react under the pressure of high volumes?

How is data retrieved for the application, how complicated will the access be? Check the execution plan of certain statement types and stored procedures. This said, have the caching of execution plans in mind when you write stored procedures (many developers don't use this built-in performance effect by making a stored procedure too dynamic !!!). A query doesn't just get faster because you use a SPROC - first it's an encapsulation and a performance boost is only reached if you understand how the Database engine can make use of caching execution plans and reusing information about the query.
Further: Don't just use an index, because you heard this is cool or a must! No way, an index is a separate 'thing' to maintain by the database. If you work on low volume datasets an index may even slow down your application. Same thing with GUID's - used as an index, they may have catastrophic consequences for your performance unless you understand the clustering methodology of your database. But anyway, in my opinion it makes mostly more sense to use another unique id for your table which is probably faster.

Don't get me wrong, I'm following the rules of database design but I also follow the experience I gained in 20 years of db design and working with tables that have millions of records and thousands of concurrent users/tasks. There're many factors that need to be considered: goals, organization details, data volume, data maintenance, data access types, error situations, etc. And I am sure I don't know as much as a specialized db admin who performs his/her job for that time span. So, as a software engineer you have to communicate a lot with the top level specialists of the other involved fields.


Don't be one-way oriented - be flexible
Consider always different data storage and retrieval possibilities. Not everything has to go into a database. I've seen real db maniacs as well as I have seen people who avoided DB's as if they were paid to do so. There are brilliant examples for data that needs to go into an XML dataset and there are definitely many situations for which it is a must to go with a database. New technology must be evaluated regarding how well it would fit into your architecture, but e.g. not just because you have heard about web services and how cool they are means that from now on everything must be done via Web Services.
For every situation there is mostly an ideal way to do it - the rest is fancy or selfish.

Modularized approach
Modularize the application. This is meant in a lot of ways. On the coding level everything should be object-oriented, communication should happen between defined interfaces of objects. The better the OO design the more time does it need to be designed, but the overhead is worth the time you spent. Then differentiate between layers of tasks, in general considered as n-tier design. Usually presentation layer, business logic and data layer. But there can be more layers in between. Some companies prefer strong abstraction layers for underlying standard layers (e.g. data access, security). Further, a layer doesn't mean that it is one block of code - it can be modularized, broken up into units (dll's, namespaces, etc.). Sometimes it's said that having many dll's may lead to a 'dll-hell' but I have not felt it that way in the past. It probably depends on the overall design of the respective project - my opinion is that when you structure your application correctly and outsource well defined units into own dll's then you can make your application more maintainable. Keep a version method around in your dll in order to be able to query the version programmatically from the application.
Modularization has many faces, there's no golden way, it always depends on the project.

First version
Don't put everything you have on your wish list into the first version of your application. Figure out what is essential. Then build that application and see how users react, how it performs in the real world - and learn from it for version 2. There are always things nobody thought where important to have and others that loose importance even if you were enthusiastic about it when first thinking of the 'wonder' application. Go step-wise.

Error logging and messages
Speaking of error handling. It's not very popular to spend time on designing a general error handling logic for one or all applications, but it's definitely worth the time.
Don't just code the application and when you feel that an error has to be thrown you display 'some' message. Think about a error module that a) logs errors into a database for later analysis, b) 'normalizes' error handling by reacting in a defined way, c) works for more than a single application, d) uses a centralized error message db ...and so on.
Such an interface costs time and money to develop but it makes life for developers more easy by having a standardized way of handling errors. Another benefit of an error log db is that you can review incidents (errors as well as warnings) centrally and create reports etc.. - quality management, so to speak.

Documentation
Nothing to say about it: do it yourself and request it from everybody else. Inline as well as external documentation is mandatory, otherwise maintenance will get quite costly later...
Using e.g. UML tools can guarantee a strict documentation process right from the beginning but with discipline it can be done also without such tools.

UML?
Opinions range from: 'absolutely mandatory instrument through all phases' to 'unnessesary crap'. I personally see it's best use on a more abstract (visualizing) level than on a detail level. Many readers will disagree now but luckily there will be at least as many supporters of my standpoint ;)

Testing of the application
Continuous testing during the entire development phase is mandatory, of course. There're are two sides, the technical and the business logic side. Normally developers and software engineers know more about the technical side and the business unit more about the business logic. It's certainly ideal if someone knows both sides. But: an application that shall be stable and reliant needs malicious and radical testers. Feed the system with the worst cases, think like a hacker but also like grandma. Type in false input wherever possible, hit keys that make no sense and test your system regarding assumptions it makes and if they can withstand in the real world.
The only thing you can rely on in an application is what you control 100% yourself - don't rely on third party data, users, etc.



Cheers, best regards,
Frank
(rev. 2006-Jan)