github

Utility module for working with github

Authors:

Jens Timmerman (Ghent University)
Kenneth Hoste (Ghent University)
Toon Willems (Ghent University)

`GithubError` ¶

Bases: Exception

Error raised by the Githubfs

`Githubfs` ¶

Bases: object

This class implements some higher level functionality on top of the Github api

`init(githubuser, reponame, branchname=None, username=None, password=None, token=None)` ¶

Construct a new githubfs object

PARAMETER	DESCRIPTION
`githubuser`	the github user's repo we want to use.
`reponame`	The name of the repository we want to use.
`branchname`	Then name of the branch to use (defaults to 'main' for easybuilders org, 'master' otherwise) DEFAULT: `None`
`username`	(optional) your github username. DEFAULT: `None`
`password`	(optional) your github password. DEFAULT: `None`
`token`	(optional) a github api token. DEFAULT: `None`

`get_path(path)` ¶

returns the path as a Github object (from agithub)

`get_repo()` ¶

Returns the repo as a Github object (from agithub)

`isdir(githubobj)` `staticmethod` ¶

Check if this path points to a directory

`isfile(githubobj)` `staticmethod` ¶

Check if this path points to a file

`join(*args)` `staticmethod` ¶

This method joins 'paths' inside a github repository

`listdir(path)` ¶

List the contents of a directory

`read(path, api=True)` ¶

Read the contents of a file and return it Or, if api=False it will download the file and return the location of the downloaded file

`walk(top=None, topdown=True)` ¶

Walk the github repo in an os.walk like fashion.

`add_pr_labels(pr, branch=GITHUB_DEVELOP_BRANCH)` ¶

Try to determine and add labels to PR.

PARAMETER	DESCRIPTION
`pr`	pull request number in easybuild-easyconfigs repo
`branch`	easybuild-easyconfigs branch to compare with DEFAULT: `GITHUB_DEVELOP_BRANCH`

`check_github()` ¶

Check status of GitHub integration, and report back. * check whether GitHub username is available * check whether a GitHub token is available, and whether it works * check whether git and GitPython are available * check whether push access to own GitHub repositories works * check whether creating gists works * check whether location to local working directories for Git repositories is available (not strictly needed)

`check_online_status()` ¶

Check whether we currently are online Return True if online, else a list of error messages

`check_pr_eligible_to_merge(pr_data)` ¶

Check whether PR is eligible for merging.

PARAMETER	DESCRIPTION
`pr_data`	PR data obtained through GitHub API

RETURNS	DESCRIPTION
	boolean value indicates whether PR is eligible

`close_pr(pr, motivation_msg=None)` ¶

Close specified pull request

PARAMETER	DESCRIPTION
`pr`	PR number
`motivation_msg`	string containing motivation for closing the PR DEFAULT: `None`

`create_gist(txt, fn, descr=None, github_user=None, github_token=None)` ¶

Create a gist with the provided text.

`create_remote(git_repo, account, repo, https=False)` ¶

Create remote in specified git working directory for specified account & repository.

PARAMETER	DESCRIPTION
`git_repo`	git.Repo instance to use (after init_repo & setup_repo)
`account`	GitHub account name
`repo`	repository name
`https`	use https:// URL rather than git@ DEFAULT: `False`

`delete_gist(gist_id, github_user=None, github_token=None)` ¶

Delete gist with specified ID.

`det_account_branch_for_pr(pr_id, github_user=None, pr_target_repo=None)` ¶

Determine account & branch corresponding to pull request with specified id.

`det_commit_status(account, repo, commit_sha, github_user)` ¶

Determine status of specified commit (pending, error, failure, success)

We combine two different things here:

the combined commit status (Travis CI sets a commit status)
results of check suites (set by CI run in GitHub Actions)

`det_patch_specs(patch_paths, file_info, ec_dirs)` ¶

Determine software names for patch files

`det_pr_labels(file_info, pr_target_repo)` ¶

Determine labels for a pull request based on provided information on files changed by that pull request.

`det_pr_target_repo(paths)` ¶

Determine target repository for pull request from given cagetorized list of files

PARAMETER	DESCRIPTION
`paths`	paths to categorized lists of files (easyconfigs, files to delete, patches, .py files)

`det_pr_title(ecs)` ¶

Create title for PR based on first easyconfigs

PARAMETER	DESCRIPTION
`ecs`	list of parsed easyconfigs

`download_repo(repo=GITHUB_EASYCONFIGS_REPO, branch=None, commit=None, account=GITHUB_EB_MAIN, path=None, github_user=None)` ¶

Download entire GitHub repo as a tar.gz archive, and extract it into specified path.

PARAMETER	DESCRIPTION
`repo`	repo to download DEFAULT: `GITHUB_EASYCONFIGS_REPO`
`branch`	branch to download DEFAULT: `None`
`commit`	commit to download DEFAULT: `None`
`account`	GitHub account to download repo from DEFAULT: `GITHUB_EB_MAIN`
`path`	path to extract to DEFAULT: `None`
`github_user`	name of GitHub user to use DEFAULT: `None`

`fetch_easyblocks_from_commit(commit, files=None, path=None)` ¶

Fetch easyblocks from a specified commit.

`fetch_easyblocks_from_pr(pr, path=None, github_user=None)` ¶

Fetch patched easyblocks for a particular PR.

`fetch_easyconfigs_from_commit(commit, files=None, path=None)` ¶

Fetch specified easyconfig files from a specific commit.

`fetch_easyconfigs_from_pr(pr, path=None, github_user=None)` ¶

Fetch patched easyconfig files for a particular PR.

`fetch_files_from_commit(commit, files=None, path=None, github_account=None, github_repo=None)` ¶

Fetch files from a specific commit.

If 'files' is None, all files touched in the commit are used.

`fetch_files_from_pr(pr, path=None, github_user=None, github_account=None, github_repo=None)` ¶

Fetch patched files for a particular PR.

`fetch_github_token(user)` ¶

Fetch GitHub token for specified user from keyring.

`fetch_latest_commit_sha(repo, account, branch=None, github_user=None, token=None)` ¶

Fetch latest SHA1 for a specified repository and branch.

PARAMETER	DESCRIPTION
`repo`	GitHub repository
`account`	GitHub account
`branch`	branch to fetch latest SHA1 for DEFAULT: `None`
`github_user`	name of GitHub user to use DEFAULT: `None`
`token`	GitHub token to use DEFAULT: `None`

RETURNS	DESCRIPTION
	latest SHA1

`fetch_pr_data(pr, pr_target_account, pr_target_repo, github_user, full=False, **parameters)` ¶

Fetch PR data from GitHub

`find_easybuild_easyconfig(github_user=None)` ¶

Fetches the latest EasyBuild version eb file from GitHub

PARAMETER	DESCRIPTION
`github_user`	name of GitHub user to use when querying GitHub DEFAULT: `None`

`find_software_name_for_patch(patch_name, ec_dirs)` ¶

Scan all easyconfigs in the robot path(s) to determine which software a patch file belongs to

PARAMETER	DESCRIPTION
`patch_name`	name of the patch file
`ec_dirs`	list of directories to consider when looking for easyconfigs

RETURNS	DESCRIPTION
	name of the software that this patch file belongs to (if found)

`github_api_get_request(request_f, github_user=None, token=None, **kwargs)` ¶

Helper method, for performing get requests to GitHub API.

PARAMETER	DESCRIPTION
`request_f`	function that should be called to compose request, providing a RestClient instance
`github_user`	GitHub user name (to try and obtain matching GitHub token if none is provided) DEFAULT: `None`
`token`	GitHub token to use DEFAULT: `None`

RETURNS	DESCRIPTION
	tuple with return status and data

`github_api_put_request(request_f, github_user=None, token=None, **kwargs)` ¶

Helper method, for performing put requests to GitHub API.

PARAMETER	DESCRIPTION
`request_f`	function that should be called to compose request, providing a RestClient instance
`github_user`	GitHub user name (to try and obtain matching GitHub token if none is provided) DEFAULT: `None`
`token`	GitHub token to use DEFAULT: `None`

RETURNS	DESCRIPTION
	tuple with return status and data

`init_repo(path, repo_name, silent=False)` ¶

Initialize a new Git repository at the specified location.

PARAMETER	DESCRIPTION
`path`	location where Git repository should be initialized
`repo_name`	name of Git repository
`silent`	keep quiet (don't print any messages) DEFAULT: `False`

`install_github_token(github_user, silent=False)` ¶

Install specified GitHub token for specified user.

PARAMETER	DESCRIPTION
`github_user`	GitHub user to install token for
`silent`	keep quiet (don't print any messages) DEFAULT: `False`

`is_patch_for(patch_name, ec)` ¶

Check whether specified patch matches any patch in the provided EasyConfig instance.

`list_prs(params, per_page=GITHUB_MAX_PER_PAGE, github_user=None)` ¶

List pull requests according to specified selection/order parameters

PARAMETER	DESCRIPTION
`params`	3-tuple with selection parameters for PRs (, , ), see https://developer.github.com/v3/pulls/#parameters

`merge_pr(pr)` ¶

Merge specified pull request

`new_branch_github(paths, ecs, commit_msg=None)` ¶

Create new branch on GitHub using specified files

PARAMETER	DESCRIPTION
`paths`	paths to categorized lists of files (easyconfigs, files to delete, patches, files with .py extension)
`ecs`	list of parsed easyconfigs, incl. for dependencies (if robot is enabled)
`commit_msg`	commit message to use DEFAULT: `None`

`new_pr(paths, ecs, title=None, descr=None, commit_msg=None)` ¶

Open new pull request using specified files

PARAMETER	DESCRIPTION
`paths`	paths to categorized lists of files (easyconfigs, files to delete, patches)
`ecs`	list of parsed easyconfigs, incl. for dependencies (if robot is enabled)
`title`	title to use for pull request DEFAULT: `None`
`descr`	description to use for description DEFAULT: `None`
`commit_msg`	commit message to use DEFAULT: `None`

`new_pr_from_branch(branch_name, title=None, descr=None, pr_target_repo=None, pr_metadata=None, commit_msg=None)` ¶

Create new pull request from specified branch on GitHub.

`pick_default_branch(github_owner)` ¶

Determine default name to use.

`post_comment_in_issue(issue, txt, account=GITHUB_EB_MAIN, repo=GITHUB_EASYCONFIGS_REPO, github_user=None)` ¶

`post_pr_labels(pr, labels)` ¶

Update PR labels

`pr_files_cache(func)` ¶

Decorator to cache result of fetch_files_from_pr.

`push_branch_to_github(git_repo, target_account, target_repo, branch)` ¶

Push specified branch to GitHub from specified git repository.

PARAMETER	DESCRIPTION
`git_repo`	git.Repo instance to use (after init_repo & setup_repo)
`target_account`	GitHub account name
`target_repo`	repository name
`branch`	name of branch to push

`reasons_for_closing(pr_data)` ¶

Look for valid reasons to close PR by comparing with existing easyconfigs.

`setup_repo(git_repo, target_account, target_repo, branch_name, silent=False, git_only=False)` ¶

Set up repository by checking out specified branch for specfied GitHub account/repository.

PARAMETER	DESCRIPTION
`git_repo`	git.Repo instance
`target_account`	name of GitHub account that owns GitHub repository
`target_repo`	name of GitHib repository
`branch_name`	name of branch to check out
`silent`	keep quiet (don't print any messages) DEFAULT: `False`
`git_only`	only use git@github.com repo URL, skip trying https://github.com first DEFAULT: `False`

`setup_repo_from(git_repo, github_url, target_account, branch_name, silent=False)` ¶

Set up repository by checking out specified branch from repository at specified URL.

PARAMETER	DESCRIPTION
`git_repo`	git.Repo instance
`github_url`	URL to GitHub repository
`target_account`	name of GitHub account that owns GitHub repository at specified URL
`branch_name`	name of branch to check out
`silent`	keep quiet (don't print any messages) DEFAULT: `False`

`sync_branch_with_develop(branch_name)` ¶

Sync branch with specified name with current develop branch.

`sync_pr_with_develop(pr_id)` ¶

Sync pull request with specified ID with current develop branch.

`sync_with_develop(git_repo, branch_name, github_account, github_repo)` ¶

Sync specified branch with develop branch.

`update_branch(branch_name, paths, ecs, github_account=None, commit_msg=None)` ¶

Update specified branch in GitHub using specified files

PARAMETER	DESCRIPTION
`paths`	paths to categorized lists of files (easyconfigs, files to delete, patches)
`github_account`	GitHub account where branch is located DEFAULT: `None`
`ecs`	list of parsed easyconfigs, incl. for dependencies (if robot is enabled)
`commit_msg`	commit message to use DEFAULT: `None`

`update_pr(pr_id, paths, ecs, commit_msg=None)` ¶

Update specified pull request using specified files

PARAMETER	DESCRIPTION
`pr_id`	ID of pull request to update
`paths`	paths to categorized lists of files (easyconfigs, files to delete, patches)
`ecs`	list of parsed easyconfigs, incl. for dependencies (if robot is enabled)
`commit_msg`	commit message to use DEFAULT: `None`

`validate_github_token(token, github_user)` ¶

Check GitHub token: * see if it conforms expectations (classic GitHub token with only [0-9a-f] characters and length of 40 starting with 'ghp_', or fine-grained GitHub token with only alphanumeric ([a-zA-Z0-9]) characters + '' and length of 93 starting with 'github_pat'), * see if it can be used for authenticated access.

github

GithubError ¶

Githubfs ¶

__init__(githubuser, reponame, branchname=None, username=None, password=None, token=None) ¶

get_path(path) ¶

get_repo() ¶

isdir(githubobj) staticmethod ¶

isfile(githubobj) staticmethod ¶

join(*args) staticmethod ¶

listdir(path) ¶

read(path, api=True) ¶

walk(top=None, topdown=True) ¶

add_pr_labels(pr, branch=GITHUB_DEVELOP_BRANCH) ¶

check_github() ¶

check_online_status() ¶

check_pr_eligible_to_merge(pr_data) ¶

close_pr(pr, motivation_msg=None) ¶

create_gist(txt, fn, descr=None, github_user=None, github_token=None) ¶

create_remote(git_repo, account, repo, https=False) ¶

delete_gist(gist_id, github_user=None, github_token=None) ¶

det_account_branch_for_pr(pr_id, github_user=None, pr_target_repo=None) ¶

det_commit_status(account, repo, commit_sha, github_user) ¶

det_patch_specs(patch_paths, file_info, ec_dirs) ¶

det_pr_labels(file_info, pr_target_repo) ¶

det_pr_target_repo(paths) ¶

det_pr_title(ecs) ¶

download_repo(repo=GITHUB_EASYCONFIGS_REPO, branch=None, commit=None, account=GITHUB_EB_MAIN, path=None, github_user=None) ¶

fetch_easyblocks_from_commit(commit, files=None, path=None) ¶

fetch_easyblocks_from_pr(pr, path=None, github_user=None) ¶

fetch_easyconfigs_from_commit(commit, files=None, path=None) ¶

fetch_easyconfigs_from_pr(pr, path=None, github_user=None) ¶

fetch_files_from_commit(commit, files=None, path=None, github_account=None, github_repo=None) ¶

fetch_files_from_pr(pr, path=None, github_user=None, github_account=None, github_repo=None) ¶

fetch_github_token(user) ¶

fetch_latest_commit_sha(repo, account, branch=None, github_user=None, token=None) ¶

fetch_pr_data(pr, pr_target_account, pr_target_repo, github_user, full=False, **parameters) ¶

find_easybuild_easyconfig(github_user=None) ¶

find_software_name_for_patch(patch_name, ec_dirs) ¶

github_api_get_request(request_f, github_user=None, token=None, **kwargs) ¶

github_api_put_request(request_f, github_user=None, token=None, **kwargs) ¶

init_repo(path, repo_name, silent=False) ¶

install_github_token(github_user, silent=False) ¶

is_patch_for(patch_name, ec) ¶

list_prs(params, per_page=GITHUB_MAX_PER_PAGE, github_user=None) ¶

merge_pr(pr) ¶

new_branch_github(paths, ecs, commit_msg=None) ¶

new_pr(paths, ecs, title=None, descr=None, commit_msg=None) ¶

new_pr_from_branch(branch_name, title=None, descr=None, pr_target_repo=None, pr_metadata=None, commit_msg=None) ¶

pick_default_branch(github_owner) ¶

post_comment_in_issue(issue, txt, account=GITHUB_EB_MAIN, repo=GITHUB_EASYCONFIGS_REPO, github_user=None) ¶

post_pr_labels(pr, labels) ¶

pr_files_cache(func) ¶

push_branch_to_github(git_repo, target_account, target_repo, branch) ¶

reasons_for_closing(pr_data) ¶

setup_repo(git_repo, target_account, target_repo, branch_name, silent=False, git_only=False) ¶

setup_repo_from(git_repo, github_url, target_account, branch_name, silent=False) ¶

sync_branch_with_develop(branch_name) ¶

sync_pr_with_develop(pr_id) ¶

sync_with_develop(git_repo, branch_name, github_account, github_repo) ¶

update_branch(branch_name, paths, ecs, github_account=None, commit_msg=None) ¶

update_pr(pr_id, paths, ecs, commit_msg=None) ¶

validate_github_token(token, github_user) ¶