Code protection for Swift Bundle Supervisor based mostly apps


Learn to collect and show code protection experiences in your Swift packages each for macOS and Linux with out utilizing Xcode in any respect.

Bitrise

The right way to take a look at utilizing SPM?

The Swift Bundle Supervisor means that you can create standalone Swift functions each on Linux and macOS. You’ll be able to construct and run these apps and you’ve got the flexibility to put in writing unit exams in your codebase. Xcode ships with the XCTest framework, however you might not know that that is an open supply library. It is out there on each single platform the place you’ll be able to set up Swift. This additionally implies that you should utilize the very same assertion strategies from the framework that you simply used to work with on iOS to unit take a look at your SPM package deal. 📦

Let me present you the best way to make a model new undertaking utilizing the Swift Bundle Supervisor:

mkdir "myProject" && cd $_


swift package deal init


swift package deal init --type=executable


Each the library and the executable template accommodates a pattern take a look at file with a dummy take a look at case. You’ll be able to run exams in some ways, there may be built-in assist for parallel execution (you’ll be able to even specify the variety of staff), you too can filter what to run by take a look at goal, take a look at case or you’ll be able to consider only one take a look at. ✅



swift take a look at


swift take a look at -l   #or `swift take a look at --list-tests`


swift take a look at --parallel


swift take a look at --parallel --num-workers 2



swift take a look at --filter myProjectTests.myProjectTests


The take a look at result’s going to look considerably like this:


Check Suite 'All exams' began at 2020-01-16 16:58:23.584
Check Suite 'myProjectPackageTests.xctest' began at 2020-01-16 16:58:23.584
Check Suite 'myProjectTests' began at 2020-01-16 16:58:23.584
Check Case '-[myProjectTests.myProjectTests testExample]' began.
Check Case '-[myProjectTests.myProjectTests testExample]' handed (0.070 seconds).
Check Suite 'myProjectTests' handed at 2020-01-16 16:58:23.654.
     Executed 1 take a look at, with 0 failures (0 surprising) in 0.070 (0.070) seconds
Check Suite 'myProjectPackageTests.xctest' handed at 2020-01-16 16:58:23.655.
     Executed 1 take a look at, with 0 failures (0 surprising) in 0.070 (0.071) seconds
Check Suite 'All exams' handed at 2020-01-16 16:58:23.655.
     Executed 1 take a look at, with 0 failures (0 surprising) in 0.070 (0.071) seconds



Processing take a look at outcomes

If it’s good to course of the result of the testing, that may be fairly difficult. I’ve created a small instrument that may convert your take a look at outcomes right into a JSON file. It is known as Testify, you’ll be able to seize it from GitHub. Let me present you the way it works:

swift take a look at 2>&1 | testify
swift take a look at --filter myProjectTests.myProjectTests 2>&1 | testify


Sadly, you’ll be able to’t use the –parallel flag on this case, as a result of in case you achieve this you may solely get progress indication as a substitute of the ultimate take a look at end result output. Luckily, you’ll be able to nonetheless filter exams, so you do not have to attend for all the things.


The swift take a look at command returns the take a look at outcomes on the usual error, as a substitute of the usual output. That is why you need to redirect the stderr into the stdout through the 2>&1 flag.


If all the things went properly you may see a pleasant JSON output, similar to this one:


{
  "endDate" : 602416925.25200009,
  "youngsters" : [
    {
      "endDate" : 602416925.25200009,
      "children" : [
        {
          "endDate" : 602416925.25200009,
          "children" : [

          ],
          "startDate" : 602416925.19000006,
          "circumstances" : [
            {
              "outcome" : "success",
              "className" : "myProjectTests",
              "moduleName" : "myProjectTests",
              "testName" : "testExample",
              "duration" : 0.062
            }
          ],
          "surprising" : 0,
          "consequence" : "success",
          "title" : "myProjectTests"
        }
      ],
      "startDate" : 602416925.19000006,
      "circumstances" : [

      ],
      "surprising" : 0,
      "consequence" : "success",
      "title" : "myProjectPackageTests.xctest"
    }
  ],
  "startDate" : 602416925.19000006,
  "circumstances" : [

  ],
  "surprising" : 0,
  "consequence" : "success",
  "title" : "Chosen exams"
}



Enabling code protection information

Code protection is a measurement of what number of strains/blocks/arcs of your code are executed whereas the automated exams are working.

I consider that protection experiences are extraordinarily helpful for the complete developer group. Mission managers can consult with the protection proportion if it involves software program high quality. The QA group may also study protection experiences & take a look at all of the remaining components or recommend new take a look at concepts for the builders. Programmers can remove many of the bugs by writing correct unit / UI exams for the applying. A protection report helps them to analyse what must be carried out as properly. Xcode has a built-in protection report web page, however you need to allow experiences first. You’ll be able to obtain the very same factor with out utilizing Xcode, by merely offering an additional flag to the take a look at command:

swift take a look at --enable-code-coverage

Okay, that is effective, however the place is my report? 🤔



The right way to show protection information?

To this point so good, you’ve generated the code protection report information, however they’re nonetheless in a extremely advanced file format. You want yet another extra instrument with a view to show them correctly.


sudo apt-get set up llvm


brew set up llvm

echo 'export PATH="/usr/native/choose/llvm/bin:$PATH"' >> ~/.zshrc

echo 'export PATH="/usr/native/choose/llvm/bin:$PATH"' >> ~/.bashrc

Now you might be prepared to make use of llvm-cov which is a part of the LLVM infrastructure. You’ll be able to learn extra about it by working man llvm-cov, however I am going to present you the best way to show some fundamental protection report for the pattern undertaking.


llvm-cov report 
    .construct/x86_64-apple-macosx/debug/myProjectPackageTests.xctest/Contents/MacOS/myProjectPackageTests 
    -instr-profile=.construct/x86_64-apple-macosx/debug/codecov/default.profdata 
    -ignore-filename-regex=".construct|Assessments" 
    -use-color

This command will generate the protection report in your exams, however provided that you’ve supplied the --enable-code-coverage flag throughout testing. It’s best to word that these llvm-cov enter paths might fluctuate based mostly in your present system. In case you are utilizing Linux, you must merely give the xctest path as a parameter (e.g. .construct/x86_64-unknown-linux/debug/myProjectPackageTests.xctest on this case), the instrument profile is situated underneath the identical listing that is not an enormous distinction, however nonetheless watch out with the platform title. Normally you do not wish to embrace the information out of your .construct & Assessments listing, however you’ll be able to specify your personal regex based mostly filter as properly. 🔍



Placing all the things collectively

You do not wish to fiddle with these parameters, proper? Neither do I. That is why I made a useful shell script that may determine all the things based mostly on the present undertaking. Save your self a couple of hours, right here is the ultimate snippet:




BIN_PATH="$(swift construct --show-bin-path)"
XCTEST_PATH="$(discover ${BIN_PATH} -name '*.xctest')"

COV_BIN=$XCTEST_PATH
if [[ "$OSTYPE" == "darwin"* ]]; then
    f="$(basename $XCTEST_PATH .xctest)"
    COV_BIN="${COV_BIN}/Contents/MacOS/$f"
fi

llvm-cov report 
    "${COV_BIN}" 
    -instr-profile=.construct/debug/codecov/default.profdata 
    -ignore-filename-regex=".construct|Assessments" 
    -use-color


It’s best to put it aside as cov.sh or one thing comparable. Add some permissions by utilizing chmod +x cov.sh and you might be able to run it by merely coming into ./cov.sh. Your protection report will seem like this:



Filename            Areas    Missed Areas     Cowl   Capabilities  Missed Capabilities  Executed       Strains      Missed Strains     Cowl-------------------------------------------------------------------------------------------------------------------------------------
myProject.swift           3                 0   100.00%           3                 0   100.00%           8                 0   100.00%-------------------------------------------------------------------------------------------------------------------------------------
TOTAL                     3                 0   100.00%           3                 0   100.00%           8                 0   100.00%

After all in case you run this script on a undertaking that has extra supply information & unit exams, it will produce a greater report. 😜



Conclusion

Utilizing take a look at outcomes and protection information is a pleasant technique to present experiences to different members in your group. By working these instructions on a steady integration server (like Bitrise), you’ll be able to automate your whole workflow.