Automated Delta Analysis

CodeScene offers the ability to detect potential maintenance problems and early warnings in your codebase. The earlier you can react to those findings, the better. That’s why CodeScene offers integration points that let you incorporate the analysis results into your build pipeline.

CodeScene’s Delta Analysis lets you catch potential problems before they are delivered to your main branch.

What are the Pre-Requisites for a Delta Analysis?

A Delta Analysis is always relative to a full analysis. CodeScene will use the latest completed analysis as a baseline for the Delta Analysis.

This is why we recommend that you configure your analysis to run at least once a day. On projects with more contributors and high commit frequencies you want to schedule CodeScene to run a full analysis each hour. We’d say that any project with more than 10 commits per day should run the analysis frequently.

Integrate CodeScene in your Continuous Integration Pipeline

CodeScene provides a REST API that lets you integrate the analysis results in a continuous integration pipeline and/or as robot comments in a code review tool like Gerrit.

CodeScene’s REST API provides a special type of analysis called a Delta Analysis. A Delta Analysis is fast, it usually just takes a few seconds to run, and is used to get early feedback on a pull request or range of new commits.

Learn what a Delta Analysis Provides

A Delta Analysis is triggered by a pull request, a range of commits or a single commit; You decide through the API. Each time you trigger a Delta Analysis, CodeScene calculates the following information:

1. Delivery risk of the suggested change set. The risk classification is relative to the risk profile for your codebase as described in Risk Analysis. Use this information to prioritize code reviews and to decide upon delivery risks.

2. Detects modified Hotspots. CodeScene attempts to rise the awareness of your change patterns by notifying your team on each pull request that modifies a top ranked Hotspot. You may use this information as a driver to refactor those Hotspots into more cohesive units, if appropriate.

3. Early detection of Complexity Trend Warnings. CodeScene already provides an early warning in case the code complexity starts to rise in a Hotspot. Now the delta analysis can catch such complexity patterns based on the changes in a pull request. That provides a great opportunity to refactor the code before delivering it to your main branch.

4. Suggests absent change patterns. This analysis identifies change sets where an expected temporal coupling is absent. If a cluster of files have changed together for a long time they are intimately related. This warning fires when such a temporal change pattern is broken. Please note that this may be good - we’ve refactored something - but it may also be a sign of omission and a potential bug. As a consequence, this warning is based on a self-correcting algorithm; If you keep ignoring the warning it will go away automatically as the temporal coupling decreases below the thresholds.

Delta analysis UI

Fig. 111 A Delta Analysis gives you early warnings and detects high risk changes.

The screenshot in Fig. 111 shows an example of a delta analysis result. Please note that this information is typically consumed and integrated via the REST API for delta analyses that we’ll discuss soon.

Please note that future releases of CodeScene will expand the Delta Analysis capabilities. Our plan is to provide even more detailed information that helps you get the most out of your time.

Use a Delta Analysis to Save Time in Code Reviews

The main advantage of a delta analysis if that it lets you react to potential problems early. But there’s a potentially large saving at the other end of the spectrum too; Instead of treating all pull requests as equals, CodeScene’s risk classification lets you prioritize your code reviews and focus your time where (and when) it’s likely to be needed the most. Code reviewer fatigue is a real thing, so let’s use our review efforts wisely.

Code review tools like Gerrit lets you select a label. For example, you specify a label that either allows or blocks the change. In addition you may select a label as an opinion (+1 and -1 in Gerrit).

When you integrate CodeScene with Gerrit, it’s our recommendation that you map CodeScene’s risk classification to an automated +1 or -1. For example, all commits below the risk category 3 may be +1, which indicates to the reviewers how much time they need to spend on this review.

In addition, the delta analyses lets you auto-detect files that seem to degrade in quality through issues introduced in the current commit or pull request. This is done by calculating code biomarkers (see Explore your Code’s Biomarkers), which are then supervised for their trend as shown in Fig. 112.

Degrading biomarkers

Fig. 112 A Delta Analysis detects degrading biomarkers.

The REST API for Delta Analyses

CodeScene lets you create a special Bot user role intended to consume the REST API. Login as administrator and create a Bot user for each of your integration points as illustrated in Fig. 113

Configure a Bot user

Fig. 113 Configure a Bot user for each of your integration points.

You trigger a Delta Analysis by POSTing a request to the REST endpoint specified in your analysis configuration as illustrated in Fig. 114.

REST API endpoint

Fig. 114 Your analysis configuration specifies the REST endpoint to trigger a delta analysis.

The payload of the POST request specifies two required fields:

  1. commits: This is a JSON array containing one or more commits. CodeScene will run a delta analysis on all these commits by considering them as a single unit of work.
  2. repository: Specifies the Git repository where the commits that you want to analyse are located. You need to specify the repository name since an analysis project may contain multiple Git repositories.

Other optional parameters:

  • coupling_threshold_percent: Specifies minimal temporal coupling for the “Absence of Expected Change” warning. Default is 80 (%).
  • use_biomarkers: Instructs CodeScene to look for degrading biomarkers. Note that this requires that the biomarkers are enabled for the analysis project.

Let’s say that we have created an analysis project by specifying a Git remote:

In this case, the repository payload parameter is PhpSpreadsheet (strip the .git extension). If we want to simulate a delta analysis of the commit designated by the hash 99e5a8e919e1f7b83371a8a586fd6d7875f63583 we issue the following request:

curl -X POST -d '{"commits": ["149f9e6"], "repository": "PhpSpreadsheet"}' http://localhost:3003/projects/64/delta-analysis -u 'CodeReview:MyPassword' -H "content-type: application/json"

You can also specify a custom temporal coupling threshold:

curl -X POST -d '{"commits": ["149f9e6"], "repository": "PhpSpreadsheet", "coupling_threshold_percent": 50}' http://localhost:3003/projects/64/delta-analysis -u 'CodeReview:MyPassword' -H "content-type: application/json"

Finally, you can enable biomarkers to detect potential code quality problems early:

curl -X POST -d '{"commits": ["149f9e6"], "repository": "PhpSpreadsheet", "use_biomarkers": true}' http://localhost:3003/projects/64/delta-analysis -u 'CodeReview:MyPassword' -H "content-type: application/json"

The example assumes that 1) CodeScene runs on localhost and 2) we have configured a user named CodeReview.

Notes to Windows Users: The curl syntax above won’t work on Windows unless you escape the payload properly. We recommend that you use Fiddler instead of curl if you want to test the API on Windows.

Once you’ve issued the POST request above, CodeScene’s Delta Analysis will respond with the following JSON document:

{"version":"1",
 "url":"/projects/64/delta-analysis/75686456d695d60d99a7cd73302f83606c8a8efc",
 "view":"/64/delta-analysis/view/75686456d695d60d99a7cd73302f83606c8a8efc",
 "result":{"risk":3,
           "warnings":[]}}

The parameters in the response carry the following meaning:

  • version: This is the version of the REST API and will change in future versions of CodeScene.
  • url: This URL points to the Delta Analysis resource in CodeScene. You can fetch it with an HTTP GET request at any time and it will return the same result document.
  • view: Points to the page in CodeScene that contains the graphical representation of the result as illustrated in Fig. 111.
  • result: This JSON object contains two fields. risk is the risk classification of the commit(s), range 1-10. warnings specify any early warnings like Complexity Trend increases. In this case it’s a low risk commit without any early warning.

Let’s close this guide by looking at a more complex result. In this case a new developer has made a modification to one of the top Hotspots on a separate branch. The Delta Analysis reports the following results:

{"version":"1",
 "url":"/projects/2/delta-analysis/43cc8a146cc0957f2fcb4b09ae3dee71d5a5cf2e",
 "view":"/2/delta-analysis/view/43cc8a146cc0957f2fcb4b09ae3dee71d5a5cf2e",
 "result":{"risk":10,
           "warnings":[
             {"category":"Modifies Hotspot",
              "details":["mvc/src/Microsoft.AspNetCore.Mvc.TagHelpers/LinkTagHelper.cs"]},
             {"category":"Absence of Expected Change Pattern",
              "details":["mvc/src/Microsoft.AspNetCore.Mvc.TagHelpers/ScriptTagHelper.cs"]}]}}

We see that CodeScene reports a high risk of 10. We also note that CodeScene calls our attention to the modified Hotspot. We use this information to review the change more carefully. Finally we note that CodeScene detects the absence of an expected change pattern. In this codebase, the LinkTagHelper.cs is usually changed together with the ScriptTagHelper.cs file. Since that wasn’t the case here, CodeScene informs us about the omission so that we can investigate it and catch a potential bug early.

Delta Analysis with Gerrit

Gerrit is a code review tool that provides a staging area for code to be reviewed. This staging area is kept separate from the main, authorative Git repository. As a consequence, the commits for a delta analysis aren’t available in the main Git repository, but in Gerrit’s mirror of the repository.

CodeScene lets you resolve this by specifying a different origin_url and a specific change_ref to fetch before the delta analysis is run. Here’s an example:

curl -X POST -d '{"commits": ["149f9e6"], "repository": "PhpSpreadsheet", "origin_url": "gerrit.mycompany.com:39429/dev/wopr", "change_ref": "refs/changes/82/577659/7"}' http://localhost:3003/projects/64/delta-analysis -u 'CodeReview:MyPassword' -H "content-type: application/json"

That is, CodeScene will fetch a specific change set from Gerrit and then run the delta analysis as indicated by the other parameters you provide.

Delta Analysis in Offline Mode

Delta analysis is triggered via an API call and thus requires authentication. Since CodeScene checks the license on a remote license server with every authentication request, the delta analysis API call will fail if CodeScene can’t reach the license server. If you don’t have an Internet connection or you don’t want to let CodeScene access the Internet, you need to specify offline-mode parameter:

curl -X POST -d '{"commits": ["149f9e6"], "repository": "PhpSpreadsheet"}' http://localhost:3003/projects/64/delta-analysis?offline-mode=offline -u 'CodeReview:MyPassword' -H "content-type: application/json"

Please note that there is a new offline-mode=offline parameter in the query string.

If you always run CodeScene in offline mode, you can also turn on Global Offline Mode in the configuration. With global offline mode, you don’t have to append offline-mode=offline parameter to your delta analysis API URLs.

Read more about the limitations and usage in Offline Mode documentation.

Recommendations for Integrating with Gerrit and CI Tools

A Gerrit of CI plugin (e.g. to Jenkins) could chose to present all the information in the Delta Analysis response. However, our recommendation is to just include a comment that specifies the risk, the number of warnings and the result view URL. That way, a user can click the URL to get more details while keeping the integration simple.