Resources | developer.brewmp.com Resources | developer.brewmp.com

Developer

resources

Using Results Manager

You can launch Results Manager from the SDK Manager Tools Page, and this window opens:

Working with workspaces

Before running tests in Test Manager, create a workspace. After creating a workspace, a database file and logs directory are generated for each workspace you create.

To create a workspace

  1. From the main window, select Workspace > Create. The Create Workspace dialog box opens.

    Note: If you create the workspace while in Administrator mode, you must also run the Test Manager command line window in Administrator mode.

  2. Under Required workspace data, enter or select the following:

    Note: You cannot change this data after creating the workspace.

    • Workspace Name
    • BVK Version
    • Platform Version
    • Target (Tip: You can change the physical target name to a more user-friendly name. See Device Settings in Target Manager Help.)
    • PID
    • OEM Build Version
  3. Under Additional data, enter or browse to the following:

    Note: You can change this data after creating the workspace. See To edit a workspace.

    • DPK path: Device pack location
    • Test data path

    The DPK and test data files are copied to the DPK folder and TestData folder, respectively, in the workspace folder.

  4. Click OK, and a database file and logs directory are created in C:\ProgramData\Qualcomm\Targets\\Workspaces\.
  5. You can now run tests in Test Manager.

To open a workspace

  1. From the main window, select Workspace > Open and the Select a Workspace dialog box opens to C:\ProgramData\Qualcomm\Targets.
  2. Navigate to the Workspaces folder and open Workspace.qwsp, the workspace file.

To edit a workspace

  1. From the main window, select Workspace > Edit and the Create Workspace dialog box opens.
  2. You can edit the following:
    • DPK path: Device pack location
    • Test data path
  3. Click OK to save your changes.

To undo or redo edits

  • After you enter a comment and click away, you can undo or redo that comment:
    • Undo: Press CTRL+Z or select Edit > Undo. The comment just entered is deleted.
    • Redo: Press CTRL+Y or select Edit > Redo. The previous undo reverts the comment.

To search a workspace

  1. From the main window, select Edit > Find, and the field displays at the bottom of the Results Manager window.
  2. Enter text and press Next and Previous to continue searching.
  3. When done, click Close.

Categorizing tests

Tests appear in the Results Manager main window under the Brew MP Feature Specification (BFS) column categorized as:

  • BL1: Tests in the Brew Level 1 (BL1) specification.
  • BL2: Tests in the Brew Level 2 (BL2) specification.
  • Deprecated: Deprecated platform version-dependent tests from legacy PVS kits.
  • NonBL: Tests that are not in the BL1 or BL2 specifications.
  • Custom names: Tests that are categorized by user-defined names.

For example:

You can create new custom BFS categories by creating your own BFS xml file containing BVK modules only. By creating your own BFS category, you can help organize, prioritize, or track testing for a specific set of tests from the BVK. Using one of the xml files as a guide (e.g., BL1BFS.xml), you can include whichever uncategorized tests (found under NonBL) that you want to show up under your custom category.

Note: If you add tests to a custom file that are already in the BL1BFS.xml or BL2BFS.xml file, the tests will appear under BL1 or BL2 because those files are parsed first.

For more information on the BFS, see the Validation Primer.

To categorize tests

  1. Find the BL1BFS.xml or BL2BFS.xml file located in C:\Program Files\Qualcomm\Brew MP SDK\Tests\\bfs.
  2. Rename the file to match your custom category name in the format BFS.xml. For example: MyTestsBFS.xml produces a new BFS category in Results Manager called MyTests, containing any tests that were included in the MyTestsBFS.xml file.
  3. Edit the file, and save it in the same location (C:\Program Files\Qualcomm\Brew MP SDK\Tests\\bfs).
  4. Close Results Manager and reopen to see your new BFS category.

Viewing test results

After tests are done running in Test Manager, you can view the tests results in the Results Manager window.

To view test results if the workspace is closed

  1. Select Workspace > Open.
  2. Navigate to the workspace (C:\ProgramData\Qualcomm\Targets\\Workspaces\\Workspace.qwsp).
  3. Select the database file for the workspace, and click Open. A new Results Manager window opens if a workspace is already open in the current window.

The following table contains decriptions of each column in the main window:
Column title or icon Description
BFS Name of the BFS category
Family Family name of Brew MP features
Subfamily Subfamily name of Brew MP features
Source Name of the kit: BVK (Brew MP Validation Kit) or PVS (Platform Validation Suite)
Module Name of the test module
Test Name of the test case
Result Specifies the reported result per test case to appear in reports generated in Report Generator
Pass Specifies the number of successful tests.
Unsu(pported) Specifies the number of unsupported tests.
Fail Specifies the number of failed tests.
Unex(ecuted) Specifies the number of unexecuted tests.
Issue Add comments about any issues.
Impact Add comments about any impact to application development.
Severity Add comments about the severity of an issue.

Note: A gear icon is placed before OAT test case names that are postprocessed (i.e., after you run the test(s) and run the script on the log to get the results); for example:

To expand or collapse test results

You can expand or collapse test results in the Results Manager window by either:

  • Using the arrows to expand or collapse individual test results one at a time.
  • Using the Edit menu to expand or collapse all test results.

To view test result logs

You can view a log of each test, except those Unexecuted, in the Results Manager window.

  1. Expand a test and click the time stamp. The log window opens.
  2. Search the log by entering keywords under Find What, and click Next or Previous.

Adding Issue, Impact, or Severity comments

Results Manager allows you to add comments under the Issue, Impact, or Severity columns. You double-click on the field and start typing. There is no limit on the maximum number of characters allowed. You can also expand the columns to read the comments as the following shows.

Importing test results

After running tests, you can import tests results in your workspace by selecting Workspace > Import. Importing results requires that the following match in the two workspaces that you merge together:

  • BVK version
  • Platform version
  • PID
  • OEM build version

Each merged result is displayed under the original result, so all test results display.

Setting preferences

You can set your preference for which result is the reported (checked) result. Test results are sorted and prioritized in this order:

  1. Pass
  2. Unsupported
  3. Failure
  4. Unexecuted

To set preferences

  1. Select Edit > Preferences. The Results Manager Preferences dialog box opens.

  2. Select from these Selection Rules:
    1. Latest result: Select the latest test result as the reported result for each test case.
    2. Best result: Select the best test result as the reported result for each test case.
    3. Latest unless Failure: Select the latest test result, unless there is a previous failure, then select the failure as the reported result.
    4. Best unless Failure: Select the best test result, unless there is a previous failure, then select the failure as the reported result.
  3. Select from these Auto-Select Behaviors:
    1. Manual selection overrides rule: Mark the manually selected result as the reported result. If no result is manually selected, the rules are used to decide the reported result.
    2. Rule overrides manual selection: Select the reported result based on the rules, and ignore any result manually selected.