REST API¶
NOTE: The REST API is currently in alpha version. This means we are likely to make breaking changes in the next release. We also plan to expand the API to make more analysis information available. Contact Empear, and we are happy to share our detailed plans and get your feedback.
You can get the following information from the API:
The access user
The rest api documentation url
Project lists
Import Project Configuration
Export Project Configuration
Single project details
Update Project by adding/removing git repository urls’
Update Project by importing development teams
Analyses list of a project
Single analysis details
Files list from analysis in descending order based on order_by value
Components list from an analysis
Single components details from an analysis
Files list from component in descending order based on order_by value
Programming language experience and distribution, both by total and by component
The REST API documentation URL¶
Using CodeScene’s REST API¶
The REST API user¶
Project lists¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects'
Import Project Configuration¶
curl -X POST --header 'Content-Type: application/json' -u username:password \
--data "@./project-config.json" 'http://localhost:3003/api/v1/projects/new'
Replace
./project-config.json
with a valid a path to your project configuration, the format is the one exported by CodeSceneMinimal example for a project where you specify the local path
./specifiy-path-project-config.json
{
"config": {
"name": "project_name",
"analysis-start-date": "2011-01-01",
"ticketidpattern": "([A-Z]{2,}-\\d+)",
"temporal-coupling-strategy": "by-ticket-id",
"repo-paths": [
"/path/to/my/project"
],
"analysis-destination": "/path/to/save/analisys"
}
}
Minimal example for a project where you specify the remote git repository urls
./specifiy-remotes-project-config.json
{
"config": {
"name": "project_name",
"analysis-start-date": "2011-01-01",
"ticketidpattern": "([A-Z]{2,}-\\d+)",
"temporal-coupling-strategy": "by-ticket-id",
"repo-paths": [
"https://my.company.com/mycompany/project1.git",
"https://my.company.com/mycompany/project2.git",
"https://my.company.com/mycompany/project3.git"
],
"gitremotelocalpath": "/path/to/save/repository/",
"analysis-destination": "/path/to/save/analisys"
}
}
Minimal example for a project where you specify the google repo url
./google-repo-project-config.json
{
"config": {
"name": "project_name",
"analysis-start-date": "2011-01-21",
"ticketidpattern": "([A-Z]{2,}-\\d+)",
"temporal-coupling-strategy": "by-ticket-id",
"local-path-for-google-repo": "/path/to/save/repository/",
"google-repo-url": "https://my.company.com/mycompany/google-repo-project.git",
"google-repo-manifest-filename": "your_manifest_name.xml",
"analysis-destination": "/path/to/save/analisys"
}
}
Export Project Configuration¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/export/configuration/json'
Replace
{project-id}
with a valid id taken from the project list results.
Single project details¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}'
Replace
{project-id}
with a valid id taken from the project list results.
Add git repository url’s¶
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -u username:password \
--data '{"urls":["https://mycompany.com/projects/p1.git", "https://mycompany.com/projects/p2.git"]}' \
'http://localhost:3003/api/v1/projects/{project-id}/repository/add'
Replace
{project-id}
with a valid id taken from the project list results.the body can contain one or multiple git repository url’s
Remove git repository url’s¶
curl -X DELETE --header 'Content-Type: application/json' --header 'Accept: application/json' -u username:password \
--data '{"urls":["https://mycompany.com/projects/p1.git", "https://mycompany.com/projects/p2.git"]}' \
'http://localhost:3003/api/v1/projects/{project-id}/repository/remove'
Replace
{project-id}
with a valid id taken from the project list results.the body can contain one or multiple git repository url’s
Update Project by importing development teams¶
curl -X POST --header 'Content-Type: text/csv' --header 'Accept: application/json' -u username:password \
--data "@./teams.csv" 'http://localhost:3003/api/v1/projects/{project-id}/teams/import'
Replace
{project-id}
with a valid id taken from the project list results.Replace
./teams.csv
with a valid a path to your teams configuration, the format is the one exported by CodeScene, first row of the file is the headerExample file
./teams.csv
author,team
max,backend
john,backend
sherri,ui
Analysis List of a project¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses'
Replace
{project-id}
with a valid id taken from the project list results.
Single analysis details¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/{analysis-id}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{analysis-id}
with a valid id taken from the analysis list results.
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest'
Replace
{project-id}
with a valid id taken from the project list results.
Files list from an analysis¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/{analysis-id}/files?page={page}&page_size={page_size}&order_by={order_by}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{analysis-id}
with a valid id taken from the analysis list results.Replace
{page}
with required page to return. If page parameter is omitted the default value is 1.Replace
{page_size}
with the number of files to return for a page. If page_size parameter is omitted the default value is 100.Replace
{order_by}
with one of the following values: “lines_of_code”, “change_frequency”, “number_of_defects”, “code_health” or “cost”. If order_by parameter is omitted the default value is “change_frequency”.
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/files?page={page}&page_size={page_size}&order_by={order_by}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{page}
with required page to return. If page parameter is omitted the default value is 1.Replace
{page_size}
with the number of files to return for a page. If page_size parameter is omitted the default value is 100.Replace
{order_by}
with one of the following values: “lines_of_code”, “change_frequency”, “number_of_defects”, “code_health” or “cost”. If order_by parameter is omitted the default value is “change_frequency”.
Components list from an analysis¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/{analysis-id}/components'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{analysis-id}
with a valid id taken from the analysis list results.
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/components'
Replace
{project-id}
with a valid id taken from the project list results.
Single components details from an analysis¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/{analysis-id}/components/{component-name}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{analysis-id}
with a valid id taken from the analysis list results.Replace
{component-name}
with a valid name taken from the components list results.
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/components/{component-name}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{component-name}
with a valid name taken from the components list results.
Component file list from an analysis¶
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/{analysis-id}/components/{component-name}/files?page={page}&page_size={page_size}&order_by={order_by}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{analysis-id}
with a valid id taken from the analysis list results.Replace
{component-name}
with a valid id taken from the component list results.Replace
{page}
with required page to return. If page parameter is omitted the default value is 1.Replace
{page_size}
with the number of files to return for a page. If page_size parameter is omitted the default value is 100.Replace
{order_by}
with one of the following values: “lines_of_code”, “change_frequency”, “number_of_defects”, “code_health” or “cost”. If order_by parameter is omitted the default value is “change_frequency”.
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/components/{component-name}/files?page={page}&page_size={page_size}&order_by={order_by}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{component-name}
with a valid id taken from the component list results.Replace
{page}
with required page to return. If page parameter is omitted the default value is 1.Replace
{page_size}
with the number of files to return for a page. If page_size parameter is omitted the default value is 100.Replace
{order_by}
with one of the following values: “lines_of_code”, “change_frequency”, “number_of_defects”, “code_health” or “cost”. If order_by parameter is omitted the default value is “change_frequency”.
Skills inventory: programming language experience and distribution¶
Successful software maintenance requires the organization to maintain system mastery of the codebase. One aspect of that mastery is to make sure you have people who know all the implementation technologies and that there aren’t any bottlenecks (e.g. “we cannot finish that feature, because our only APL programmer is on a 3 months vacation”). CodeScene’s skills inventory analysis provides input to system mastery discussions and planning.
In general, CodeScene answers the following questions:
How many developers know language X?
Who are those developers, and what are their relative contribution experience in that technology?
The analysis info is available both for the whole codebase as well as for specific sub-systems and components. Use this information as input to discussions on:
Do we need to train more people in a specific programming language?
Do we have to hire people for a specific programming language?
Do we have enough C#/C++/Java/Python programmers in that critical sub-system where we plan lots of new features?
Before we look at the specific endpoints, lets explain the contribution fields that you will find in the responses:
In combination with CodeScene’s on- and off-boarding simulation module, you can also use this information to see the effects of rotating staff between different sub-systems or components and identify key personnel on a technical level.
There are two main REST endpoints:
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/experience/languages'
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project-id}/analyses/latest/experience/languages/{language}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{language}
with the name of a programming language, e.g. JavaScript, Python, etc.
Both of these endpoints also exist on a component level. The response format is identical, only limited to programming languages and contribution statistics withing the given component:
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project_id}/analyses/latest/components/{component}/experience/languages'
curl -X GET --header 'Accept: application/json' -u username:password \
'http://localhost:3003/api/v1/projects/{project_id}/analyses/latest/components/{component}/experience/languages/{language}'
Replace
{project-id}
with a valid id taken from the project list results.Replace
{language}
with the name of a programming language, e.g. JavaScript, Python, etc.Replace
{component-name}
with a valid id taken from the component list results.
Examples: Use Cases and Scripts¶
We use CodeScene’s REST API ourselves, and in this section we’d like to share some of the scripts we use. These scripts are particularly useful on large codebases as a basis for custom reports.
These examples are based on the following strategy:
Invoke the REST API using curl.
Parse the reply with the jq tool.
Pipe the response to other shell functions when needed.
As such, everything below is built using existing command line tools.
Let’s start by defining a helper function for calling the REST API:
call_api(){ curl -sS -u api:secret http://localhost:3003/api/v1$1 ${@:2}; }
export -f call_api
Using the call_api utility, we can now query the REST API for specific purposes as illustrated in the following paragraphs.
Project Statistics¶
List project names and ids:
call_api /projects | jq -r '.projects[] | [.name,.id] | @csv' | sort
Calculate the total lines of code in the 10 most changed files:
call_api /projects/1/analyses/latest/files | jq -r '.files | sort_by(.change_frequency)
| .[0:10] | .[].lines_of_code' | paste -sd+ - | bc
Maintenance and Housekeeping¶
List all local repository paths for all projects (a bit more complex since we call the API in a loop):
call_api /projects | jq -r '.projects[].id' | while read id;do call_api /projects/$id \
| jq -r '.repository_paths_on_disk[]'; done | sort -u
File-Level Analysis Information¶
List the top hotspot files by change frequency:
call_api /projects/1/analyses/latest/files\?order_by=change_frequency | jq -r '.files[]
| [.change_frequency,.name] | @csv'
Get the current code health score + the number of defects for the top 25 hotspot files of a project:
call_api /projects/1/analyses/latest/files\?order_by=change_frequency\&page_size=25 \
| jq -r '.files[] | [.change_frequency,.code_health.current_score,.number_of_defects,.lines_of_code,.name] | @csv'
Code Health Analysis¶
List the 100 most worked on files together with their code health trend, and sort by relevance:
call_api /projects/1/analyses/latest/files\?order_by=change_frequency\&page_size=100 \
| jq -r '.files[] | [.change_frequency,.code_health.current_score,.code_health.month_score,.code_health.year_score,.name] | @csv'
Code Health Rules¶
The /metadata endpoint lets you access global rules and information. Here’s how you retrieve a list of all the code health rules that you can come across in a Virtual Code Review:
call_api /metadata/code-health-rules \
| jq -r '.code_health_rules[] | [.code_health_rule] | @csv'
Architecture-Level Analysis Information¶
List a project’s architectural components sort by name:
call_api /projects/1/analyses/latest/components | jq -r '.components | sort_by(.name)| .[] | [.name, .ref] | @csv'
Get some metrics for the architectural component by name in a project:
call_api /projects/1/analyses/latest/components/`read -p "Enter component name: " name && echo $name` \
| jq -r '[.age,.change_frequency,.lines_of_code,.system_health.current_score,.name] | @csv'
Get some metrics for all architectural components order by name in a project :
call_api /projects/1/analyses/latest/components | jq -r '.components | sort_by(.name)| .[] | .name' \
| xargs -I % bash -c "call_api /projects/1/analyses/latest/components/% | jq -r '[.age,.change_frequency,.lines_of_code,.system_health.current_score,.name] | @csv'"
Create New Project¶
CodeScene is using the following environment variables when creating a new project, for Tomcat deployment, you can also use JNDI Context properties
-
CODESCENE_ANALYSIS_RESULTS_ROOT
¶
“/path/to/analysis/”
-
-
CODESCENE_CLONED_REPOSITORIES_ROOT
¶
“/path/to/repositories/”
-
Example of create a new project using git repo URL and using environment variables
call_api /projects/new -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' \
-d '{ "config": { "repo-paths": ["https://github.com/my-project/my-project-1.git"] } }'
This will create a project with name ‘my-project-1’
Example of create a new project using git repo URL without setting environment variables, you need to provide in the config the “gitremotelocalpath” and “analysis-destination”
call_api /projects/new -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' \
-d '{ "config": { "repo-paths": ["https://github.com/my-project/my-project-2.git"], \
"gitremotelocalpath": "/path/to/repositories/", \
"analysis-destination": "/path/to/analysis/"} \
}'
This will create a project with name ‘my-project-2’