Release of XStudio 1.0b1

The version 1.0b1 of XStudio has been released and is available for download at the usual location: http://www.xqual.com/support/download.html.

Main changes from 1.0a4 are:

* Several minor bug-fixes
* History buttons previous/next (browser-like)
* Attachments on requirements and specifications
* Attachments inheritance on req, spec and tests
* Charset for country_id in table_absence
* Updated User's Guide

XStudio 1.0: Browser-like Previous/Next buttons

With the introduction of some anchors in different parts of XStudio, the user is often redirected to one specific node of one specific tree. It was then becoming to be able to go back and forth through the history and navigate like within a browser.

Hence, I implemented this feature. This will be available in XStudio 1.0b1.

XStudio 1.0: Inheritance of the attachments

Starting from XStudio 1.0b1, attachments are inherited through their whole hierarchy. For instance, if you attach a file (i.e. a mockup design) to a requirement, you will be able to see in this requirement's "attachments" tab not only this attachment but ALL the attachments associated to his ancestors (parent folders and category).

Of course, you can directly open or download the attachment from any of the ancestor nodes but you can also move directly to one of these parent by clicking on the anchors:

Website updated

The website (http://www.xqual.com) has been updated to support all the popular browser:

• Firefox 3+
  
• IE7+
• Safari
• Opera

I also included a "share toolbar" including:

• RSS
• Blog
• Twitter
• StumbleUpon
• Facebook
• MySpace
• Digg
• Delicious
• Reddit
• Google Bookmarks
• Reddit
  

Release of XStudio 1.0a4

The version 1.0a4 of XStudio has been released and is available for download at the usual location: http://www.xqual.com/support/download.html.

Main changes from 1.0a3 are:

* Coverage metrix on campaigns (full and partial)
* Coverage metrix on campaign sessions (full and partial)
* Coverage metrix on category (req + spec + test)
* Coverage metrix on folders (req + spec + test)
* Automatically create testcases if the launcher is requesting it
* Progression/Regression metrix on campaign
* Bug-fix:
- Percentage rounding in coverages
- Dependency graph modality problems
- VisualStudio launcher results parsing finally implemented

I also worked on:
* New website (http://www.xqual.com) supporting perfectly IE7+, Firefox3+, Chrome and Safari
* New open-source section (GPL3) on the website containing all the launchers
* Initiation of teh development of automated functional tests on:
- the XStudio NSIS installer (using the AutoIt launcher)
- the XStudio Java unit testing (using the JUnit4 launcher)

In addition, I developed a bunch of new launchers for XStudio:
* Froglogic Squish for Qt
* Froglogic Squish for the web application
* Microsoft Visual Studio Team System
* Jalian Marathon
* Beanshell
* JUnit v3
* JUnit v4
* PyUnit
* NUnit (.NET)
* TestNG

XStudio 1.0: Automatically create the default testcase

In some specific cases, the notion of testcase is useless. For instance a JUnit test will match one test class. This test class can run several methods which could be identified to testcases. The problem is that it is impossible to run only one single method within a JUnit class (using the console runner JUnitCore).

However, without testcase, XStudio will not execute anything. So, the user would need to create one fake testcase for each test. Hence, the launcher would ignore the testcase index, would execute the test including all its methods, retrieve the results, traces etc. and store all this in the fake testcase. But creating a default testcase manually for each test is a bit painful and unnecessary from a user point of view.

In these cases, the launcher can now call (i.e. in the initialize() method) the method setDefaultTestcaseMustBeCreated(true). This will automatically create a default testcase at runtime if no testcase exist.

XStudio 1.0: New famous launchers are ready

Since I'm almost ready with the features of XStudio v1.0, I started working on some famous launchers people will like:
- Beanshell
- Froglogic Squish for Qt
- Froglogic Squish for Web
- JUnit v4
- JUnit v3

All are implemented and just need a little bit more of testing.

An example of integration with JUnit v4:

All the results/messages are detailed enough and include the major information from the JUnit console messages.

In addition, the complete output from JUnit is stored as an attachment to the testcase, so that the user can get the complete log if necessary:

I'll also work on a launcher for SQL and possibly CUnit this week.

XStudio 1.0: New progressions/regressions statistics

A new "Statistics" tab will be present on any campaign on XStudio 1.0a4.

The information are taken from all the sessions executed belonging to the campaign. All the results are put in a graph so that the evolution (progression or regression) in time is clearly visible. The last session's results are also displayed.The new tab is split in 2 tabbed panes: one that shows the evolution of tests results and another one going deeper at the testcase level.


This feature will be available in XStudio 1.0a4 at the usual location http://www.xqual.com/support/download.html (probably in just a few days).

XStudio 1.0: New coverage metrics

XStudio 1.0a4 (that was not planned but hey I'm a bit ahead of time so...) will include a new great feature that I've just finished to develop: in addition to all the already existing coverage metrics provided, some new ones will appear in the next version. Coverage from ANY folder in the requirement or specification trees (this allows to refine metrics on one particular node in the tree) and detailed coverage on campaigns and campaign sessions.

Specification tree:
How many specifications under the selected folder are covered by some tests.
Of course, you can get the details of which specifications are covered and which are not.

Requirement tree:
How many requirements under the selected folder are covered by some specifications.
How many requirements under the selected folder are covered by some tests (through specifications).
Of course, you can get the details of which requirements are covered by some specifications and which are not but also which requirements are covered by some tests.

Coverage of a campaign:
Since a test campaign is by definition a group of test, we can now also get the following metrics:
- Specifications fully covered by the campaign
- Specifications partially covered by the campaign
- Specifications not covered by the campaign
- Requirements fully covered by the campaign
- Requirements partially covered by the campaign
- Requirements not covered by the campaign


Coverage of a campaign session:
On the campaigns, all the data data were global (based the complete set of requirements and specifications). We can get some even more interesting data on campaign sessions since the session is associated to a SUT (and an SUT is associated to a list of requirements).

Hence, we can again refine the information and get the following metrics:
- Specifications associated to the SUT fully covered by the campaign
- Specifications associated to the SUT partially covered by the campaign
- Specifications associated to the SUT not covered by the campaign
- Requirements associated to the SUT fully covered by the campaign
- Requirements associated to the SUT partially covered by the campaign
- Requirements associated to the SUT not covered by the campaign

This allows to know precisely if some tests are missing in the campaign, or if some associations between requirements, specifications and tests are missing.

I think those new features are great tools to help analyzing what you're testing and more importantly what you're missing. They will be available in XStudio 1.0a4 at the usual location: http://www.xqual.com/support/download.html.

Release of XStudio 1.0a3

The version 1.0a3 of XStudio has been released and is available for download at the usual location: http://www.xqual.com/support/download.html.

Main changes from 1.0a2 are:

* Countries management
* Absences management (per user)
* Holidays management (per country)
* Week-ends management (per country)
* Better presentation of the calendars (new colors, legend,
borders etc.)
* Remember scale + expended nodes when moving in time in the calendars
* Full implementation of calendars with absences, holidays,
personalized week-ends (depending on the country)
* Add several new user profiles:
- HR Manager
- Developer
- Test Operator
* Add windows7 and snow leopard in OS list
* Print stack traces each time a "SQL lost error" is thrown
* For sprint velocity, use the most used week-end settings among
all users
* WeekDaySelect formitems using now Standard identification for days
of the week (former schedules to review!)
* Add SUT-Requirement association including:
- Inheritance of requirements (from SUT)
- Inheritance of tests (from SUT)
* Factorization of a LOT of SQL functions (LIST and FIRSTRESULT)
* Add parameter feature.not_implemented_testcase_notification.enabled
* Added a couple of launchers:
- AutomatedQA TestComplete 7
- TCL
* Improved security of CRunner class
* Open-source section on the website including SDK and all sources
* Bug-fix:
- Tree retrieval when one single element belongs to the root node
- Forbid points outside the sprint dates in the velocity charts
- Localization fixing
- Update global hashtables when updating user's country
- Remove useless logs

XStudio 1.0: New configuration parameter

When you run a campaign session and that the campaign included some testcases marked as not-implemented, they are skipped and a message is popup at the end of the complete execution to remind the user so. It may be annoying in some cases (it blocks further automatic campaign executions until the user presses OK), so I added a new configuration parameter in xstudio.conf and xagent.conf:

feature.not_implemented_testcase_notification.enabled

If you set this parameter to 0, this warning message will never show up.
By default, this parameter is set to 1.

XStudio 1.0: SUT to requirements association

One of the features lots of people asked me for a while was the ability to associate a SUT with a set of requirements so that the complete flow looks like:

SUT <--> Requirements <--> Specifications <--> Tests <--> Campaign <--> Session

To do so, when creating a SUT, a new Requirements tab allows to select a list of requirements in the requirement tree.


However, creating a new SUT is a task that happens quite often, so re-selecting all the appropriate requirements at each new version would be fastidious. To fix this problem, I implemented a simple way to "import" all requirements associated with another SUT (i.e. the previous version).

Hence, imagine you formerly created a SUT "myProduct v1.0" and associated it with a hundred of requirements. You've run a number of test campaigns successfully on this SUT but now a "myProduct v1.1" is released with a few new features (so new requirements).

The process is quite simple: on the Requirements tab: press the "Preset import settings" button, you will be prompted to select a source SUT, then press the "Preset import" button: all importable requirements from the SUT you selected will be highlighted with a specific flags on the right column and automatically checked by default. However, you can still manually remove or add manual new requirements (i.e. the new requirements of "myProduct v1.1"):


That's it! Of course, all these features are also available while editing a SUT.
This will allow me to add new interesting information in the campaign for a better coverage analysis (I will do this a little bit later).

This new feature will be available in XStudio 1.0a3.

XStudio 1.0: Absences and Holidays management implemented

I've just finished implementing the absences and holidays management system. Each user is now associated to one location/country (I'm using ISO convention).


Each country can be easily configured: public holidays can be added, edited and deleted. In addition, each country is associated to some week-ends settings. Hence, most of the countries in the world are using Saturday and Sunday for the week-ends but some others (such as Israel, Qatar, Algeria etc.) are using Friday and Saturday and again some others (such as Saudi Arabia) Thursday and Friday.

The default settings are supposed to be correct but it's good to let the administrator of the system customizes it if necessary. These settings are accessible through the Settings menu entry.


Hence, all the calendars are affected by this change as each user may have different week-ends.

Absences are also included in the calendars so you have a complete picture of the projects or the staff workload. Status are then easier to figure out.

Half-yearly scale

Monthly scale

Note that the calendar tree is still expandable as usual. This greatly facilitates the reading i.e. when you want to know the details about why a user is overloaded in a certain time frame.

A legend (including some gradient colors for the workload) has been added to help the reading .
These features will be availabe in XStudio 1.0a3 at the usual location: http://www.xqual.com/support/download.html.

Release of XStudio 1.0a2

The version 1.0a2 of XStudio has been released and is available for download at the usual location: http://www.xqual.com/support/download.html.

Main changes from 1.0a1 are:

* Report with test ordered like during the execution

* Complete control over manual tests execution through a control bar:

- Previous Test

- Previous Testcase

- Rewind Testcase

- Pause execution

- Resume execution

- Next Testcase

- Next Test

* Timeout setting per test while executing manually (using attributes)

* Bug-fix:

- Sometimes some elements were hidden in the trees when they were the only child of their container

XStudio 1.0: Timing constraints set per test for manual testing

As you already know, while executing manual testing, the launcher provides instructions to the operator to execute actions (and do verifications). The launcher is also configurable so that some timing restrictions can be added. For instance, it is possible to give 10 minutes maximum for the operator to execute the actions/verifications (i.e. you need to test that a message appears on the screen of the SUT in less than 10 minutes; if the operator doesn't validate that he saw that message within the 10 minutes then the test is automatically set as failed).

The issue is that this configuration was at the launcher level - which means that all the tests had necessarily the same timing constraints. This was a huge drawback and I've just corrected it.

There is no more timing configuration to set for the whole campaign session when creating it. However, there is now a much more flexible way to handle timeout for manual tests: by default all the tests belonging to a category associated to manual or simple_manual launchers have no timeout. If you want to set some specific timeouts to a test you just need to associate to this test one or several of the following attributes (with a value for the timeout):

• timeout test description (ms): maximum delay to validate the test description
• timeout test prerequisites (ms): maximum delay to validate the test prerequisites
• timeout testcases execution (ms): maximum delay to execute any actions or verifications implied by the testcases in this test

When manually executed, each test will be applied with their specific timeout values.

Without timeout

With timeout

Note also, that all these attributes are dynamic. This means that when creating the campaign session, you can overwrite any of these default values (on each test if necessary) to customize one campaign session execution:

This new feature will be available in XStudio 1.0a2 that will be delivered around the 13th of September at this location: http://www.xqual.com/support/download.html.