This page was an attempt to get converstation started about the state and nature of structure software development where I worked, now it is public. As a weather person doing pre-dominately programming I've had to learn many concepts on my own and have yet to find one method usable in all circumstances. Furthermore I've been unable to find a methodology applicable to the structure of programs by a team of one; i.e. software development in industry as I understand it is now a team process. As an individual, I'm finding the process inefficient and at times frustrating. Thus I continue to look for techniques and tools to make my job easier. Below is a collection of information and links I've found at this time. Hopefully, this will be expanded upon and will become a springboard for improvement(s) of the processes I use.
Documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing. – Dick Brandon
It keeps amazing me that intelligent, cooperating people can't see the value of documenting what needs to be documented. Without it we would still be carving kernel code in cave walls. – Unknown
The palest ink is better than the best memory. – Chinese Proverb
If it's not documented it doesn't exist. – Jon Schiefelbein
The number one rule I have for software developement is document everything. The second rule for software development I have is document everything and document it first. The third rule of software development I have is document everything and talk about it, let everyone who will read it, read it. The importance of documentation can NOT be overstated. Documentation is NOT easy and should not be treated as an after thought.
A couple of years after writing the information present above and below I continue to struggle with documentation. General observation: it is NOT easy. A recent reading, enlightened me to the fact I'm not alone in my struggles.
For documentation, Sphinx has become the primary tool for documenting Python projects. Sphinx has been used for general documentation, several web sites and a couple of books are written entirely using Sphinx. However, for documenting other languages in a relatively literate programming fashion (as it does for Python) Sphinx does not have the tools at this time. Using Sphinx for documenting Python has lead to a few best practices and an understanding of why the methodology is used in industry.
Three documents MUST exist for any project regardless of language. These documents do NOT need to be overly formal or lengthy. The documents actually benefit both the programmer and the manager as they enhance communication and ensure everyone is thinking similar thoughts and talking apples to apples. The three documents required are:
THE SOURCE for what the program should do. This should be plainly written in very simple terms; i.e. the program will … or the user will input variable y. Without this document there is no proof as to what the program should do, what output is expected and what input is required. This lack of proof creates animosity between the user and the developer. The user will have requirement creep and keep requesting more and more features and functionality. The programmer/developer will assume they are done when in fact they did not even meet one of the requirements. You will wind up with a situation from Dilbert. Users, managers and developers must read and approve this document BEFORE any actual coding begins. Let me repeat myself because this is important: users, managers and developers must read and approve this document BEFORE any actual coding begins.
There are two intended audiences for the analysis and design documentation: peer developers and managers. The documentation for managers will be the simplest and contain the least amount of technical terms and jargon, an executive summary mind set and writing style would be effective here. Peer developers need more detailed discussion on the design and it will be assumed the knowledge level is very similar to the developer. Talk of abstract classes and object linking is appropriate and desired in this section. More importantly the developer/designer needs to discuss why the design decisions were made and the benefits and limitations of these decisions. This is important not only for peer developers but for managers as it will allow them to consider risk. Please see the articles listed below for a more detailed discussions.
This document is necessary to prove the program does what is required. It should show in some form input and then program output resulting from the input. There should also be some lint type output to demonstrate the code was written to some standard and a profiled run indicating how the program performs. The profile output should indicate how many resources each section of the program consumed, either in CPU time, memory size or disk space.
Unit testing is a (critical) component of the Agile Development methodology. Unit testing is a very powerful way of insuring code is bug free and performs as expected. It can also reduce refactoring time since the developer can change the code and still perform the same tests to ensure the output did not change. A better, albeit longer, explanation is available on Wikipedia along with a history of the concept of test driven development.
Unit testing has it most significant support with Java but all significant languages and all languages (including SAS) I have been exposed to have unit testing frameworks. Python's PyUnit (a.k.a. unittest) is built in and there is also Nose. Java has the well known JUnit. C and C++ both have unit testing libraries and it is possible to find open source unit testing libraries for FORTRAN. A somewhat comprehensive list is available at: http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks. Google is a great place to start learning the semantics of the testing frameworks for your language of choice.