quarkslab/bgraph

BGraph is a tool designed to generate dependencies graphs from Android.bp soong files.

BGraph

BGraph is a tool designed to generate dependencies graphs from Android.bp soong files.

Overview

BGraph (for Build-Graphs) is a project aimed at create build graphs from blueprints in AOSP and querying those graphs.

In short, this project builds/uses Unified Dependency Graph for the Android Open Source Project by parsing and linking modules defined in the Android build system Soong.

Use-cases

You should use this tool if you want to find:

• all the dependencies of a source file in AOSP;
• all the sources involved in the building of a target in AOSP;
• common dependencies between two targets.

Usage

% bgraph --help                                                       
Usage: bgraph [OPTIONS] COMMAND [ARGS]...

  BGraph - generate and query build dependency graphes.

  BGraph is used to manipulate build dependency graphs generated from
  blueprint files. The main commands are:

      - generate : used to generates multiples graphs

      - query: used to query a previously generated graph

  To get more help, see the online documentation.

Options:
  -v, --verbose         Activate verbose output  [default: False]
  --install-completion  Install completion for the current shell.
  --show-completion     Show completion for the current shell, to copy it or
                        customize the installation.

  --help                Show this message and exit.

Commands:
  generate         Generate BGraph's from a mirror dir.
  generate-single  Generate a BGraph from a branch.
  list             List the BGraph already generated.
  query            Query a BGraph.

Installation

Using poetry

poetry install bgraph

Using pip

pip install bgraph

Using docker

docker build -f docker/Dockerfile -t bgraph .

This will create a container with git, repo and bgraph and will take some time (because it compiles git from the source).

See Docker for more instructions.

Prerequisites

• python3.8

Optional dependencies for the builder:

• repo
• git (>25): since we're using partial-checkouts, a modern version of git is required
• at least 1Go of free disk space
• (Optional: AOSP mirror)

See Building from AOSP for more details.

Documentation

Documentation

Licence

Apache-2

Contributing

Contributions are always welcome!

See the Contribution documentation for details on all you need to know about contributing.

Authors

• dm ([email protected])

bgraph's Issues

Unable to generate

Generate

Steps to reproduce

mkdir test && \
cd test && \
mkdir bgraphs && \
python3.8 -m venv venv && \
source venv/bin/activate && \
pip install bgraph && \
bgraph generate bgraphs 'https://android.googlesource.com/'

Error Log

2021-12-13 16:39:17,862 - bgraph.builder.builder - INFO - Found 1174 branches
2021-12-13 16:39:17,863 - bgraph.builder.builder - INFO - Start composing for android-1.6_r1.1_
2021-12-13 16:39:27,625 - bgraph.builder.builder - ERROR - Unable to init the repository. Verify that the mirror is correct.
2021-12-13 16:39:27,625 - bgraph.builder.builder - INFO - Start composing for android-1.6_r1.2_
2021-12-13 16:39:41,370 - bgraph.builder.builder - ERROR - Unable to init the repository. Verify that the mirror is correct.
2021-12-13 16:39:41,370 - bgraph.builder.builder - INFO - Start composing for android-1.6_r1.3_

...

get-poetry.py file not found at Docker build

poetry remove get-poetry.py script: python-poetry/poetry@82eb934

it cause docker build failed:

Removing intermediate container 90e4c278acbc
 ---> 024f1beaa7d7
Step 13/30 : RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 &&     curl -sSL https://storage.googleapis.com/git-repo-downloads/repo > /opt/repo/repo &&     chmod a+x /opt/repo/repo /opt/poetry/bin/poetry
 ---> Running in 85ea8f413bfd
  File "<stdin>", line 1
    404: Not Found
             ^
SyntaxError: invalid syntax

Can this be used to get list of all dependencies?

I've made a graph and I'm able to query dependencies of specific files, but my end goal really is to create SBOM (software bill of materials) of aosp and for that I would need the full list of all dependencies of the project. Is that possible to query with the structure of bgraph?

Support local clones?

Hey. Thank you for your tool. I was just trying to use it to understand the dependency graph of a local clone of the master-art manifest but am unable to generate a graph for it.

I believe bgraph expects an entire local mirror (which I don't have) or you have to provide a mirror URL (which is too time consuming and eats a lot of disk space, specially if I already have a local clone). Maybe I've misunderstood how the tool works, but I couldn't get it to generate a graph due to the missing manifest.git file.

I think it would be immensely useful if there was an option to effectively just do a find /path/to/local/android/clone -name Android.bp and then create a graph from those files as it avoids the above issue.

For reference, I'm just following the official documentation on how to build ART (https://android.googlesource.com/platform/art/+/refs/heads/master/build/#building-as-a-module-on), which recommends to use the master-art manifest.

EDIT: Ah apologies. It does seem like I've misunderstood how the tool works. You can specify the branch so the master-art manifest is supported.