Results 1 to 2 of 2

Thread: STC should exit with non-zero exit (errorlevel) code when tests fail, and 0 for pass

    Success! Looks like we've fixed this one. According to our records the fix was applied for ORION-1136 in 2.0.0.
  1. #1
    Sencha Premium User
    Join Date
    Jan 1970
    Posts
    14

    Default STC should exit with non-zero exit (errorlevel) code when tests fail, and 0 for pass

    All CI (build systems) like Jenkins, GitlabCI, and others, expect to run commands, and then examine their errorlevel (shell exit code) to see if the tests passed or not. This allows us to interrupt the flow of a build, and mark it red.

    This is not working for us. STC will always exit(0) at the end and does not exit(1) or any other value when a test fails.

    This needs to be fixed.


    1. Write failing test.
    2. Execute it with STC
    3. On DOS command prompt type echo %ERRORLEVEL%
    4. Expected: not 0
    5. Actual 0

    Corrollary is that if you write a PASSING test we expect 0 error code at exit.


    As a secondary bug I would like to point out that when a test fails, I expect to see a line number of the test where the failure occurred, so that I don't have to guess which expect failed out of ten of them that are comparing a bool to another bool value. The message "Expected false to equal true" (an unfortunate practice inherited from Jasmine it seems) is useless, so unless we can add a .because('reason') at least I might expect the failed test to output:


    chrome52 - Chrome 53.0.2785.116 / Windows 10.0
    WorklistControllerSpec.js
    onShowHideWorklistClick function: be able to show/hide Worklist view
    Expected false to equal true (line 182)

    Instead all we get is:

    chrome52 - Chrome 53.0.2785.116 / Windows 10.0
    WorklistControllerSpec.js
    onShowHideWorklistClick function: be able to show/hide Worklist view
    Expected false to equal true

  2. #2
    Sencha - Test Dev Team
    Join Date
    Nov 2011
    Location
    Boca Raton, FL
    Posts
    75

    Default

    Thanks for the report! I have opened a bug in our bug tracker.

Similar Threads

  1. Replies: 3
    Last Post: 18 Dec 2015, 3:54 PM
  2. Replies: 1
    Last Post: 21 Oct 2014, 3:23 AM
  3. Replies: 7
    Last Post: 11 Nov 2013, 5:06 AM
  4. Replies: 11
    Last Post: 7 Oct 2013, 9:50 AM
  5. Close or exit application from code.
    By alohainc in forum Sencha Touch 2.x: Q&A
    Replies: 4
    Last Post: 2 Jan 2013, 9:22 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •