Integrate Costs and Issues into CodeScene (Jira, Trello, Azure DevOps and GitHub Issues)

CodeScene’s provides an optional integration to Jira, Trello, Azure DevOps or GitHub Issues. The integration has the following pre-requisites:

  • Issue numbers are included/referenced in the commit messages.

  • You use labels and/or issue types to distinguish different kinds of work (e.g. Bugs, Features).

When present, CodeScene’s Jira, Trello, Azure DevOps and GitHub Issues integrations let you measure:

  • Accumulated costs per hotspot and sub-system.

  • Trends by work type, such as “Planned” versus “Unplanned” work.

CodeScene’s cost analyses let you reason about the technical and organizational findings from a financial perspective. For example, how much time do you spend on defects in your top hotspots? What amount of work is unplanned? And what happens over time?

CodeScene supports multiple cost models depending on the data you have available such as cycle times, story points, or time spent. CodeScene can deduce costs automatically based on your development history, so you don’t even need to have developers reporting their time to be able to get a detailed view of your development costs. We cover all options and provide our recommended setup in this guide. Let’s start by looking at the overall model.

CodeScene’s Cost Model

CodeScene offers a breakdown of the development costs on three separate levels: file-, architecture-, and system-level. The file level corresponds to the hotspots, the architecture level accumulates costs on a component/service/module level, whereas the system level presents the cost trend as aggregated for all application code.

CodeScene calculates cost trends on a file-, architecture-, and system-level.

Fig. 49 CodeScene calculates cost trends on a file-, architecture-, and system-level.

Using CodeScene’s cost model, you get the specific development costs on each hotspot. Use this data to inform re-work decisions and to communicate the costs of technical debt to the business:

CodeScene calculates cost per hotspot, including both feature and defect data.

Fig. 50 CodeScene calculates cost per hotspot, including both feature and defect data.

For larger hotspots, it might not be realistic – nor financially wise – to refactor a whole module. For that purpose, CodeScene comes with its X-Ray analysis that lets you break down the defect statistics to a detailed function level.

CodeScene breaks down defect statistics from a hotspot file to a function level.

Fig. 51 CodeScene breaks down defect statistics from a hotspot file to a function level.

Calculating Development Costs: the four options

There are four strategies, and you need to select one of them depending on what data you have in Jira, Trello, Azure DevOps or GitHub Issues:

  • Estimated development time: calculate a cycle time for the number of hours from the time an issue entered a specific Jira state until the last commit is done on this issue. The cycle time is then adjusted for the number of hours in a typical work day. That is, a cycle time of 3 days has a cost of 3 * 8 hours. In addition, CodeScene distributes the total costs proportional to the change impact when multiple modules/files are modified for the same issue. This cost model lets CodeScene calculate costs as opposed to rely on time reported in Jira, which is often incorrect, incomplete, or both.

  • Issues: the number of issues associated with a given file or architectural component. Use Issues to get a summary and inspect general trends, but without assessing actual time spent.

  • Points: the cost of an Issue in /Story points/. Use this option if you keep track of story points and use them to communicate within the organization.

  • Minutes: the cost expressed in time. This method requires that time has been reported on the specified cost-field in JIRA. Use this option if you have accurate data in JIRA on how much time you have spent on each issue.

Export Cost Data to Excel/CSV

All cost trends in CodeScene provides a button that lets you export to a CSV file and explore further details in a spreadsheet application.

NOTE: The exported data use the internal cost units. For issues and story points, this unit is always the unit of presentation in the graphs. However, for time-based cost units like Estimated development time or Minutes, the exported cost is in minutes. This typically differs from the values presented in the graphs since CodeScene converts them to more suitable visual presentations (e.g. hours, days).

Export CodeScene's cost data to CSV.

Fig. 52 Export CodeScene’s cost data to CSV.

Configuration

Connect CodeScene to Jira

Jira is enabled and configured per project. Navigate to the “PM Integration” tab in your project’s configuration and select “Jira”:

Start by selecting "Jira"

Fig. 53 Start by selecting “Jira”

Fill in your Jira credentials here. We recommend using a Jira API token as the password. Note also that a separate token for accessing GitHub is necessary when selecting the option to extract issue numbers from GitHub Pull Requests.

You also have to specify a cost model that determines how CodeScene calculates the costs. The cost model options are described earlier in this document.

Once you press “Save and Continue”, you are presented with the detailed configuration options. See further below in this document for a detailed walkthrough of the configuration options.

Connect CodeScene to Trello

Trello is enabled and configured per project. Navigate to the “PM Integration” tab in your project’s configuration and select “Trello”:

Configure the information you want to retrieve from Trello.

Fig. 54 Configure the information you want to retrieve from Trello.

You need an API Key and an API token from Trello. You generate those credentials via the Trello app-key.

You also have to specify a cost model that determines how CodeScene calculates the costs. The cost model options are described earlier in this document.

Once you press “Save and Continue”, you are presented with the detailed configuration options, as specified above.

Connect CodeScene to Azure DevOps

Azure DevOps is enabled and configured per project. Navigate to the “PM Integration” tab in your project’s configuration and select “Azure DevOps”:

Configure the information you want to retrieve from Azure DevOps.

Fig. 55 Configure the information you want to retrieve from Azure DevOps.

You need an access token from Azure Devops. See Authenticate access with personal access tokens. Assign the Code:Read (to read pull requests data) and Work Items:Read scopes.

You also have to specify a cost model that determines how CodeScene calculates the costs. The cost model options are described earlier in this document.

Once you press “Save and Continue”, you are presented with the detailed configuration options, as specified in the next section.

Connect CodeScene to GitHub Issues

GitHub Issues are enabled and configured per project. Navigate to the “PM Integration” tab in your project’s configuration and select “GitHub Issues”.

You need an access token from GitHub with repo permissions. See Creating a token.

Once you press “Save and Continue”, you are presented with the detailed configuration options, as specified in the next section. Note that the GitHub Issues integration supports only the Issues cost model.

Specify the Detailed Configuration Options for Jira, Trello, Azure DevOps and GitHub Issues

Configure the information you want to retrieve from Jira.

Fig. 56 Configure the information you want to retrieve from Jira.

External Projects: Select one or more Jira projects that CodeScene will use as data sources.

Cost Field: is used for he cost models Points and Minutes, Select the Jira field that has your cost estimates. If you use Issues or Estimated development time as cost model, then it is not necessary to configure Cost Field and you can leave it as is.

Work In Progress Transition Name: Specify the name of the Jira status that indicates that the development work has started. Often, this is the “In Progress” or “In Development” state.

The Work In Progress Transition Name is relevant for two analyses: 1. If you use Estimated development time as cost model, then this field is mandatory. 2. Delivery Performance: if you have enabled the delivery performance module, then this config option is needed to calculate lead times.

Excluded Work Types For Jira, all issues found on commits are used in analysis. Work types can still be excluded by specifying them here.

Supported Work Types For GitHub, Azure and Trello, all issues of the worktypes you specify here will be included in analysis.

Defect Work Types: specify the JIRA labels or JIRA Issue Types that will be regarded as defects. This configuration is used to calculate defect densities and work type trends.

The Rename Work Types field allows the work types to be mapped to different analytical categories that you can define yourself. How you do this depends on the type of analysis you wish to perform.

Tip: Rename Work Types to distinguish Planned and Unplanned work

When looking at cost trends, the most interesting distinction is typically between Planned- versus Unplanned Work.

By specifying Rename Work Types option, the Jira labels are translated to the specified label before being sent to CodeScene. For example, if your Jira project contains Feature and Documentation labels, like in the illustration above, these can be categorized together as Planned Work, while Bug and Defect are treated as Unplanned Work; the Refactoring label – which doesn’t have a translation – will be sent as is. Mapping labels this way can allow you to see more meaningful trends. You are free to map labels however you like depending on your analytical needs.

Advanced Information

CodeScene’s Cost Distribution

CodeScene’s cost model contains a number of features that reduce bias in the available cost data. For example, let’s say that you have a Jira issue with the known cost of 100 hours. In that issue, the developer spent 95% of the time working on one complex hotspot, and then just made a simple tweak to a config file. How should those 100 hours be split between the impacted files? Assigning “100 hours” as cost to each file is clearly the wrong thing to do as it exaggerates the development cost of that simple config file.

For this purpose, CodeScene distributes the calculated costs for a specific issue on two orthogonal levels: 1) across impacted modules, and 2) across the relevant types of work.

Distribution across modules: CodeScene distributes the derived costs proportional to the change impact when multiple modules are modified as part of the same issue. That is, the modules with most work in a specific issue get proportionally higher costs than modules that had less work. CodeScene automatically deduces this distribution.

Distribution across issue types: Since a Jira/Trello issue can have multiple issue types and labels, CodeScene distributes the costs equally across all referenced types of work for the specific issue. This is relevant to the cost trends in CodeScene. For example, let’s say we have a cost of 100 hours for a specific issue and entity. The issue has two labels in Trello: Feature and UI. CodeScene will distribute the costs so that each of those types of work, Feature and UI, get 50 hours as their costs.

NOTE: The cost unit issues is a special case since issues are never split; it doesn’t make sense to say that the cost of a specific module is “3.25 issues”. Instead CodeScene selects the first known issue type for that issue and assigns it the cost.