Tiers
Obviously, not all software will meet all standards. The enforcement
of standards is a politcally sensitive issue, and ultimately is the
responsibility of the Principal Investigator (PI) of each software
project. For academic projects, software engineering techniques
really only work if the PI agrees that the standard or tool is
valuable, and allocates time towards using it.
We can use a set of software standards to rate individual projects, which
gives collaborators an idea of the stability of a package.
For each of the standards, we could have four tiers, Red, Yellow, Green and Blue, where Blue is the highest level of compliance.
Software with low levels of compliance is not necessary bad software, it
might be software under development by only a few authors, or it might
be a project that is just starting out.
| Standard |
Red |
Yellow |
Green |
Blue |
| UML |
UML diagrams never used |
UML diagrams or similar aids sometimes used |
UML diagrams available for all source code |
UML diagrams shipped with source code |
Reviews |
No Reviews |
Design and code reviews sometimes used |
Design and code reviews regularly performed and available to developers online |
Software not released until design and code reviewed |
| Platforms |
Runs under one platform |
Runs under two platforms |
Runs under both Unix and Windows NT |
Runs under all platforms |
| Version Control |
Release Tree is not in version control |
Release Tree is under version control of some sort |
CVS |
CVS Notification via email |
| Style Guide |
No style guide |
Documented style guide. No style checker tools |
Documented style guide. Style checker tools available |
Documented style guide. Style checker tools run on nightly build |
| Build System |
No Build System |
Custom, one off build system using custom scripts |
Build system uses configure and makefiles |
Documented build system. ./configure; make; make install works |
| Documentation |
No documentation |
Installation docs available, some user and programmer documentation available |
Complete installation, user and programmer documentation available |
Complete installation, user and programmer documentation available, uses source based documentation system such as javadoc |
| Testing |
No test suite |
Regression test suite available |
Tests run every night with code coverage reports |
Tests run every night with code coverage reports and 100% method test coverage |
| Nightly Builds |
No Nightly Build |
Nightly Build |
Nightly Build with test release built |
Test release created and built, test suite run, code coverage reports generated |
Ratings for various software projects
Probably the least controversial way to rate software is to have
the authors rate themselves and see where they stand.
(Note that jars.com
provides ratings by a set of moderators)
Below is a set
of ratings for various software projects, where the packages
are listed by start date, with the oldest package listed on the left.
| Standard |
Ptolemy Classic |
Tycho |
Ptplot |
Ptolemy II |
Diva |
| Start Date |
1992 |
9?/1995 |
10/1997 |
1/1998 |
6/1998 |
| Lines of code |
420K |
110K |
20K |
115K |
66K
|
| Language |
C/C++/Tcl |
Tcl/Itcl |
Java |
Java |
Java |
| UML |
Red |
Red |
Yellow |
Yellow |
Yellow |
| Reviews |
Red |
Red |
Green |
Green |
Yellow |
| Platforms |
Yellow |
Blue |
Blue |
Blue |
Green |
| Version Control |
Yellow |
Yellow |
Green |
Green |
Blue |
| Style Guide |
Yellow |
Yellow |
Green |
Green |
Yellow |
| Build system |
Blue |
Yellow |
Green |
Blue |
Green |
| Documentation |
Green |
Blue |
Blue |
Blue |
Yellow |
| Test Suite |
Yellow |
Yellow |
Yellow |
Blue |
Red |
| Nightly Build |
Green |
Green |
Red |
Blue |
Red |
|
