ReleaseTestingProcess

bitweaver Release Testing Process

Created by: system, Last modification: 13 Dec 2004 (04:43 UTC) by SEWilco
The Testing Process (PRELIMINARY)
In most cases, testing of bitweaver release candidates (RC) for release, will be an iterative process. Each iteration is called a testing cycle.
  1. Developers agree on the scope of changes for a release branch (in cvs)
  2. Developers make changes to the bitweaver code base to fix bugs and add enhancments
  3. Developers collectively agree that all planned changes and enhancements have been made to a planned release code branch.
  4. Developers have performed due diligence in testing their changes and enhancements and believe that the new / altered bitweaver is ready for release to users (customers).
    • Testers are not lackeys for developer superstars. Developers should do their own “engineering testing” and do their best to make their changes “ready for prime time”
    • Testers may (and often will be) developers. Try not to test your own work. If you can think of a test case for your fix / enhancement, why wouldn’t you have already coded for it?
  5. Developers and the test coordinator agree on the scope of the testing effort and develop some sort of test plan that will limit testing to a scope appropriate to for the changes made to the code base.
  6. Just before a round of release candidate testing has started (RCnn),
    1. The cvs development branch is labeled to reflect the release candidate’s version and RC number
    2. The bitweaver testing.site ( http://testing.bitweaver.org ) is configured to provide a shared a testbed for testers to validate correct bitweaver behavior (as well as demonstrate / verify anomalous behavior).
      • Cvs updates are performed manually just before testing each candidate release and is not updated again until just before a subsequent candidate release
      • Testing.bitweaver.org is a linux / postgres – based resource provided for RC testers
      • Because bitweaver is operated on multiple server platforms and operated from multiple browser clients, testing effort is multiplied by the number of web servers, web browsers, and database management systems on which it is to be supported.
      • If you are providing your own hardware for testing a release candidate, create a separate, pure cvs “sandbox” used only for testing purpose (just like for http://testing.bitweaver.org )
  7. The candidate's installation procedure should be tested by installing through each of the major configurations, so at least installations are likely to succeed.
      1. A webserver installation at webserver root (files owned by a www user).
      2. A user installation at webserver root (files owned by a user, permissions for webserver).
      3. A user installation in a user directory.
  8. Armed with the test plan, the testing coordinator coordinates RC testing efforts by whatever means deemed effective by him / her. Suggested communication methods include:
    1. Announcement of an RC testing cycle / delivery freeze via development mailing list email (includes pleadings for testing effort participation)
    2. Creation of an RCnn signup / status wiki page on www.bitweaver.org
    3. Coercion or of any / all people in the #bitweaver IRC channel
  9. If and when testing reveals a failure to behave as intended, a failure report is generated and communicated back via the bitweaver SourceForge Bug web page ( http://sourceforge.net/tracker/?group_id=101599&atid=630083 )
  10. The lead developer monitors RC bug reports on SourceForge and assigns them to the appropriate developer.
  11. Testing continues until one or more of the following occur:
    • The nature of the fix to a reported bug is so extensive that it threatens to invalidate further testing without a cvs update
    • The fix of a reported bug is in a piece of shared code (e.g.: library) that is shared by numerous packages
    • The test plan is executed to the end (with some bugs reported)
    • The test plan is executed to the end without exception (no bugs)
  12. If there were bugs, the next release candidate test cycle begins again at step #5, above (RC number increments).
  13. If there were no bugs, then the successful release candidate is labeled by version.


Comments

Not really smart from this

by Kozuch, 21 Feb 2007 (21:45 UTC)
(:confused:)