Contributing#

Hey thanks for considering contributing to blackbench! It’s awesome to see you here. Anyway, there’s a lot of ways you can contribute to blackbench, whether that’s opening a constructive bug report or feature request, writing docs, to actually writing some code.

Setting up development environment#

Requirements#

CPython 3.8 or higher
Nox

All development commands are managed by Nox which provides automated environment provisioning. For us, it’s basically a task runner. I strongly recommend using pipx.
pre-commit [optional]

pre-commit runs Git commit hooks that assert a baseline of quality. Once again, pipx is encouraged.

Steps#

Fork the blackbench project on GitHub if you haven’t already done so.

Clone your fork and cd into the resulting directory.

dev@example:~$ git clone https://github.com/${USERNAME}/blackbench.git
Cloning into 'blackbench'...
remote: Enumerating objects: 244, done.
remote: Counting objects: 100% (244/244), done.
remote: Compressing objects: 100% (141/141), done.
remote: Total 244 (delta 85), reused 212 (delta 58), pack-reused 0
Receiving objects: 100% (244/244), 154.29 KiB | 360.00 KiB/s, done.
Resolving deltas: 100% (85/85), done.
dev@example:~$ cd blackbench
dev@example:~/blackbench$

Add the fork’s parent (ie. upstream) as an additional remote.

dev@example~/blackbench$ git remote add upstream https://github.com/ichard26/blackbench.git

If pre-commit is available, install the Git pre commit hooks.

dev@example:~/blackbench$ pre-commit install
pre-commit installed at .git/hooks/pre-commit

Run the setup-env session with Nox.

You’ll use this virtual environment to run blackbench from source (ie, during manual testing). Flit’s editable install feature will be used so don’t worry about recreating the environment after changing something, they should be instantly reflected!
```
dev@example:~/blackbench$ nox -s setup-env
nox > Running session setup-env
nox > /home/dev/.local/pipx/venvs/nox/bin/python -m virtualenv /home/dev/blackbench/venv
nox > /home/dev/blackbench/venv/bin/python -m pip install flit
nox > /home/dev/blackbench/venv/bin/python -m flit install --deps production --symlink
nox > Virtual environment at project root named `venv` ready to go!
nox > Session setup-env was successful.
```
Important

The environment created by setup-env is only for manual testing. Automated testing or other development commands should be done via the sessions configured in noxfile.py. This is why the created virtual environment only has the dependencies needed to run blackbench.

Activate the created virtual environment.

Unix/macOS

Bash

dev@example:~/blackbench$ . venv/bin/activate
(venv) dev@example:~/blackbench$

Zsh

dev@example:~/blackbench$ . venv/bin/activate.zsh
(venv) dev@example:~/blackbench$

Fish

dev@example:~/blackbench$ source venv/bin/activate.fish
(venv) dev@example:~/blackbench$

Windows

C:\Users\dev\blackbench> venv\Scripts\activate
(venv) C:\Users\dev\blackbench>

Celebrate! … and then get to work on that change you’ve been thinking about :)

Development commands#

As already mentioned, Nox is basically a task runner here. Most development commands are already configured in noxfile.py so running them correctly is easy as can be.

All the Nox sessions should support both -r and -R, so if the sessions are too slow (especially for rapid iteration during development), try of them. The only major exception is setup-env but that one doesn’t use a Nox-provisioned environment anyway.

Also, it’s possible to pass extra arguments to any the sessions’ main command. Want to add -k "not provided" to the pytest run in tests? That’s possible via nox -s tests -- -k "not provided".

Testing#

To run the test suite, just run the tests session:

$ nox -s tests

If you want to collect coverage too, there’s the tests-cov sessions for that:

$ nox -s tests-cov

Nox has one more awesome feature and that’s making it really easy to run the test suite against multiple versions of Python at once. Matter of fact, the commands above will make Nox run the test session for every supported Python version it can find. You can select a single version by prepending the version (eg. nox -s tests-3.9).

Tip

If you need to run the test suite with a specific version of Black you can use the --black-req option. Eg. nox -s tests-3.8 -- --black-req "black==21.5b2". Note that the -- is important since the option was implemented at the session level and is 100% custom.

Linting#

Calling pre-commit to run linters is as simple as:

$ nox -s lint

Docs#

There’s two sessions for documentation, which one you choose depends on what your goal is. If you’re looking to do a complete and clean (re)build of the documentation, just run the aptly named docs session:

$ nox -s docs

BUT, if you’re actively making changes to the documentation, having it automatically rebuild and refresh on changes will make your life easier. That’s available using:

$ nox -s docs-live

Once the first build has been completed, there should be a link that serves the built documentation. As mentioned, the page will automagically refresh on changes!

PR guidelines#

To make it easier for all of us to collaborate and get your PR merged, there’s a few guidelines to be noted:

If your PR has user-facing changes (eg. new target, bugfix), please add a changelog entry.
Your PR should try to maintain excellent project coverage, although this isn’t a blocker.
Please include an explanation for the changes (and maybe a summary too if complicated enough) in the commit message.
If CI fails, please address it, especially if the test suite failed since compatibility with muliple systems and environments must be maintained.
You should expect somesort of response within three days. If not, feel free to ping @ichard26.

Getting help#

If you get stuck, don’t hesistate to ask for help on the relevant issue or PR. Alternatively, we can talk in the #black-formatter1 text channel on Python Discord. Here’s an invite link!

Appendix A: area-specific notes#

Adding a new target#

I’m looking for two kinds of targets: “normal” and “micro” . Normal targets should represent real-world code (so the benchmarking data actually represents real-world performance). Micro targets should be small and are focused on one area of Black formatting (and mostly exist to measure performance in a specific area, like string processing).

In terms of guidelines: normal targets shouldn’t be bigger than ~2000 lines (this is to keep time requirements to run the benchmark based off the target manageable), and micro targets shouldn’t be bigger than ~400 lines. Oh and for micro targets, make sure their focus hasn’t been already covered by another target.

Release process#

Before you fear what lies in front of you please know that the release process was designed to be simple and lightweight. The fact you’re doing one in the first place is awesome and your time should be treated well! So in pursuit of that, here’s the blackbench release process:

Note

You don’t have to follow these steps carefully, they’re more like guidelines that aim to make the 99% case easy. I’m sure there’s situations this release process won’t work and in that case, just use your best judgement.

Once you’ve decided that a release is due, please verify the following things:
- the changelog has at least one entry (unless you’re doing a post-release or something like that)
- CI for the main branch is all green
If you don’t have a local development environment, either setup one up or just make sure you have flit2 ready to go
Checkout main and/or cleanup your local repository
Run flit -s do-release -- <version> with this release’s version. The Nox session will handle the rest by:
- checking the local repository is reasonably clean
- updating both the version string and changelog to include the new version and today’s date
- commiting those changes and then tagging the commit
- checking out the repository in a temporary isolated directory
- running the flit publish command
- updating the version string and changelog again for development
- and finally commiting those changes
Push the newly created commits and tag to the GitHub repository
Go get a coffee or something, you just did a release! Congrats!

1: I know it’s specifically for Black, but blackbench is a development tool for Black so I consider it acceptable - although I never asked … but then again, I am a maintainer of Black so yeah :p
2: I’d strongly recommend also setting up pre-commit so any dumb mistakes by the release automation is caught before release, but the release automation shouldn’t be buggy so it’s your call.