kanhari/gpt-commit-summarizer

gpt-commit-summarizer

See announcement blogpost.

Don't have time and want to get hacking right away? Check out the Getting Started section.

• Getting Started
• Troubleshooting
• Encountered any bugs?

The gpt-commit-summarizer GitHub Action is a powerful tool that harnesses the capabilities of OpenAI's state-of-the-art text-davinci-003 large language model to provide summaries of the changes introduced by a pull request in a repository. By generating the git diff for each commit and for each modified file and sending it to the OpenAI API with a carefully crafted prompt, the action is able to produce concise and informative summaries that can greatly enhance collaboration and understanding in large codebases.

The action then performs a higher level call to the OpenAI API to generate a summary of the entire pull request, from the summaries of individual commits and file differences. This summary is then posted as a comment on the pull request.

Getting Started

Automatic Setup

Works for all git repositories. follow the CLI instructions:

npx add-gpt-summarizer@latest

Note: Requires that you have Node.js installed.

Setting up

To use this action, you will need to have an OpenAI API key. If you don't already have one, you can sign up for an OpenAI API key here.

Once you have your API key, you will need to add it to your GitHub repository as a secret. To do this, go to your repository's settings and navigate to the "Secrets" section. Click on "Add a new secret" and enter the secret name OPENAI_API_KEY and the value of your API key.

Next, you will need to add the workflow file to your repository. Create a file named .github/workflows/gpt-commit-summarizer.yml (relative to the git root folder) and copy the following code into it:

name: GPT Commits summarizer
# Summary: This action will write a comment about every commit in a pull request, as well as generate a summary for every file that was modified and add it to the review page, compile a PR summary from all commit summaries and file diff summaries, and delete outdated code review comments

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  summarize:
    runs-on: ubuntu-latest
    permissions: write-all  # Some repositories need this line

    steps:
      - uses: KanHarI/gpt-commit-summarizer@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

This workflow file tells GitHub to run the action whenever a new pull request is opened or updated.

That's it! You're now ready to use the gpt-commit-summarizer action in your repository. Each time a pull request is opened or updated, the action will automatically generate a summary of the changes made in each commit, add a summary for every file that was modified to the review page, compile a PR summary from all commit summaries and file diff summaries, and delete outdated code review comments.

Troubleshooting

I have heard some unverified reports that the OpenAI API may block requests from the IP addresses of some runners. If you encounter this issue, you can try using a self-hosted runner to run the gpt-commit-summarizer action. This can be done by setting up a runner on a server that you control, and then adding the runner to your repository.

To set up a self-hosted runner, you will need to follow these steps:

• Install the GitHub Actions Runner on your server. Follow the instructions in the documentation to do this.

• Add the self-hosted runner to your repository. Follow the instructions in the documentation to do this.

• Modify the workflow file to use the self-hosted runner. Open the .github/workflows/gpt-commit-summarizer.yml file and add the runs-on field to specify the self-hosted runner that you want to use. For example:

name: GPT Commits summarizer
# Summary: This action will write a comment about every commit in a pull request, as well as generate a summary for every file that was modified and add it to the review page, compile a PR summary from all commit summaries and file diff summaries, and delete outdated code review comments

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  summarize:
    runs-on: self-hosted
    permissions: write-all  # Some repositories need this line

    steps:
      - uses: KanHarI/gpt-commit-summarizer@master
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Encountered any bugs?

If you encounter any bugs or have any suggestions for improvements, please open an issue on the repository. Alternatively, you can contact me at my email.

License

This project is licensed under the MIT License.

gpt-commit-summarizer's Issues

Non nodejs install?

Some way to install this in a repo without installing nodejs would be wonderful.. Possible to just fork a repo with it setup already and edit the config? Not really sure about github actions details..

Request: Option for more condensed summary

This is in many ways super userful, so thanks for that 🙌 One thought, it would be useful with less verbose output or fewer comments i.e. having only one summary. Would also be great if updates to PRs would close or delete outdated summaries.

Append the total token used

Feature request: Append the total token used at the end of the summary.

Quick test reports incorrect diff summary

saw this on reddit (cool project btw 😄), thought I'd provide feedback from a quick test I ran today in a personal repo. I introduced the following diff (adding a key to a python dict):

-            'F64': 'double'
+            'F64': 'double',
+            'F128': '_uint128_t'

but the summary was:

Added F128 to the dictionary of type mappings.
Changed the type mapping for F64 from double to _uint128_t.

but F64's value wasn't changed, F128 was only introduced.

Not sure what else to add here. Thought it might be useful for further tuning the query sent to openai.

Not able to write the summary automatically

Follow the steps detailed, but need help to see the requested result. When the action gets called, it reports 404 error:
Request failed with status code 404. Looks like the API call to a resource is not available for some reasons. Could you help in this case please. Thanks

GPT summary:

Error: couldn't generate summary

PR summary so far:

Error: couldn't generate summary

Action logs:
OpenAI file summary prompt for 2023-12-19/BinaryGap/Program.cs:
You are an expert programmer, and you are trying to summarize a git diff.
Reminders about the git diff format:
For every file, there are a few metadata lines, like (for example):

diff --git a/lib/index.js b/lib/index.js
index aadf691..bfef603 100644
--- a/lib/index.js
+++ b/lib/index.js

This means that lib/index.js was modified in this commit. Note that this is only an example.
Then there is a specifier of the lines that were modified.
A line starting with + means it was added.
A line that starting with - means that line was deleted.
A line that starts with neither + nor - is code given for context and better understanding.
It is not part of the diff.

The following is a git diff of a single file.
Please summarize it in a comment, describing the changes made in the diff in high level.
Do it in the following way:
Write SUMMARY: and then write a summary of the changes made in the diff, as a bullet point list.
Every bullet point should start with a *.

THE GIT DIFF OF 2023-12-19/BinaryGap/Program.cs TO BE SUMMARIZED:

@@ -1,10 +1,15 @@
-namespace BinaryGap
+using System.ComponentModel;
+
+namespace BinaryGap
 {
     internal class Program
     {
         static void Main(string[] args)
         {
             //convert number to binary
+            //Just adding this comment to test git
+            //Add more comments here
+            //Add more testing here
             Console.WriteLine(Result(9));
         }
 

writable: true,
destroyed: false,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
maxRequestsOnConnectionReached: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: true,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
strictContentLength: false,
_contentLength: 1603,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
_closed: false,
socket: TLSSocket {
  _tlsOptions: [Object],
  _secureEstablished: true,
  _securePending: false,
  _newSessionPending: false,
  _controlReleased: true,
  secureConnecting: false,
  _SNICallback: null,
  servername: 'api.openai.com',
  alpnProtocol: false,
  authorized: true,
  authorizationError: null,
  encrypted: true,
  _events: [Object: null prototype],
  _eventsCount: 10,
  connecting: false,
  _hadError: false,
  _parent: null,
  _host: 'api.openai.com',
  _closeAfterHandlingError: false,
  _readableState: [ReadableState],
  _maxListeners: undefined,
  _writableState: [WritableState],
  allowHalfOpen: false,
  _sockname: null,
  _pendingData: null,
  _pendingEncoding: '',
  server: undefined,
  _server: null,
  ssl: [TLSWrap],
  _requestCert: true,
  _rejectUnauthorized: true,
  parser: null,
  _httpMessage: [Circular *1],
  [Symbol(res)]: [TLSWrap],
  [Symbol(verified)]: true,
  [Symbol(pendingSession)]: null,
  [Symbol(async_id_symbol)]: 1[8](https://github.com/serumhyb/ProgrammingPractice/actions/runs/7639785575/job/20813537931?pr=2#step:2:9)0,
  [Symbol(kHandle)]: [TLSWrap],
  [Symbol(lastWriteQueueSize)]: 0,
  [Symbol(timeout)]: null,
  [Symbol(kBuffer)]: null,
  [Symbol(kBufferCb)]: null,
  [Symbol(kBufferGen)]: null,
  [Symbol(kCapture)]: false,
  [Symbol(kSetNoDelay)]: false,
  [Symbol(kSetKeepAlive)]: true,
  [Symbol(kSetKeepAliveInitialDelay)]: 60,
  [Symbol(kBytesRead)]: 0,
  [Symbol(kBytesWritten)]: 0,
  [Symbol(connect-options)]: [Object],
  [Symbol(RequestTimeout)]: undefined
},

Request: lite version that only rewrites commit messages

I work mainly as a solo developer, as such I rarely pull request for myself. It would be great if this would simply write my commit messages for me without having to make a pull request.

Very large diff returns an error

I got an "Error: OpenAI query too big" testing it with a large xml file with a bunch of changes.

Maybe limit the diff to a reasonable size and include a note that it was trimmed?

(Very cool by the way)