googleapis / java-bigquerystorage Goto Github PK

View Code? Open in Web Editor NEW

55.0 55.0 78.0 8.42 MB

License: Apache License 2.0

Java 99.77% Python 0.02% Batchfile 0.01% Shell 0.20%

java-bigquerystorage's Introduction

Google BigQuery Storage Client for Java

Java idiomatic client for BigQuery Storage.

Quickstart

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.35.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-bigquerystorage</artifactId>
  </dependency>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-bigquerystorage</artifactId>
  <version>3.4.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.36.0')

implementation 'com.google.cloud:google-cloud-bigquerystorage'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-bigquerystorage:3.4.0'

If you are using SBT, add this to your dependencies:

libraryDependencies += "com.google.cloud" % "google-cloud-bigquerystorage" % "3.4.0"

Authentication

See the Authentication section in the base directory's README.

Authorization

The client application making API calls must be granted authorization scopes required for the desired BigQuery Storage APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the BigQuery Storage API calls.

Getting Started

Prerequisites

You will need a Google Cloud Platform Console project with the BigQuery Storage API enabled. You will need to enable billing to use Google BigQuery Storage. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

Installation and setup

You'll need to obtain the google-cloud-bigquerystorage library. See the Quickstart section to add google-cloud-bigquerystorage as a dependency in your code.

About BigQuery Storage

BigQuery Storage is an API for reading data stored in BigQuery. This API provides direct, high-throughput read access to existing BigQuery tables, supports parallel access with automatic liquid sharding, and allows fine-grained control over what data is returned.

See the BigQuery Storage client library docs to learn how to use this BigQuery Storage Client Library.

Samples

Samples are in the samples/ directory.

Sample	Source Code	Try it
Json Writer Stream Cdc	source code
Parallel Write Committed Stream	source code
Storage Arrow Sample	source code
Storage Sample	source code
Write Buffered Stream	source code
Write Committed Stream	source code
Write Pending Stream	source code
Write To Default Stream	source code

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Transport

BigQuery Storage uses gRPC for the transport layer.

Supported Java Versions

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

For new development

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Keeping production systems current

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Legacy support

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all patches.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

Where to find specific information

The latest versions and the supported Java versions are identified on the individual GitHub repository github.com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

Versioning

This library follows Semantic Versioning.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version	Status
Java 8
Java 8 OSX
Java 8 Windows
Java 11

Java is a registered trademark of Oracle and/or its affiliates.

java-bigquerystorage's People

Contributors

Stargazers

Watchers

Forkers

renovate-bot pmakani kmjung stephaniewang526 athakor shollyman bigang-li rahulkql xiaozhenliu-gg5 allenc3 yangnuoyu noahdietz henry1417 twinkle-kadia isabella232 yayi-google chethi007 veronicawasson gnanda jacobstocklass tryweirder whitemike889 emkornfield j143 esert-g suztomo sridivakar dorina-sada ramasltu stephenjudkins jhambleton burakgok ebyhr jphalip ruparee tom-s-powell haritha99 prash-mi noak2 githassan tomoponzoo sandhusanthakumar tracyz-g yirutang agrawal-siddharth alicejli 1e0ng risnayak surinder07 reuvenlax weifonghsia subham611 egreco12 rohankumardubey mutlurenegado monikarajendran01 rdebokx roy-tc jun-liu-10 oleksiyivanov arturowczarek gaolemeng suryasoma le-dai yifatgortler shubhwip mina-asham snuyanzin komukomo vteja11 orossini sarkarbikram90 norlove anthavio tomrussello brianseung-google

java-bigquerystorage's Issues

Synthesis failed for java-bigquerystorage

Hello! Autosynth couldn't regenerate java-bigquerystorage. 💔

Here's the output from running synth.py:

Cloning into 'working_repo'...
Switched to a new branch 'autosynth'
Cloning into '/tmpfs/tmp/tmpola62m03/googleapis'...
Cloning into '/tmpfs/tmp/tmpola62m03/synthtool'...
Switched to branch 'autosynth-self'
Note: checking out 'b74c0cea992e88654df9b52fdc6872bb44132238'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at b74c0ce chore: regenerate README (#232)
Note: checking out 'f8a9933e5e98202b04ef427f28d1d79221190fa4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at f8a9933 chore(java): allow overriding the versioning section of the Java README (#495)
Note: checking out '28e76243c23cc282efbb288cb558c174e3e5e9ee'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 28e76243 feat: integrate gapic generator change in googleapis
Switched to a new branch 'autosynth-self-2'
2020-04-30 13:58:08 [INFO] Running synthtool
2020-04-30 13:58:08 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
    from synthtool import update_check
  File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 13:58:08 [ERROR] Synthesis failed
HEAD is now at b74c0ce chore: regenerate README (#232)
Switched to branch 'autosynth-self'
Note: checking out 'b74c0cea992e88654df9b52fdc6872bb44132238'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at b74c0ce chore: regenerate README (#232)
Previous HEAD position was f8a9933 chore(java): allow overriding the versioning section of the Java README (#495)
HEAD is now at 6b685a2 fix: synthtool path (#515)
Previous HEAD position was 28e76243 feat: integrate gapic generator change in googleapis
HEAD is now at 7e1c7603 Add an OAuth scope annotation in build_service proto file
Switched to a new branch 'autosynth-20'
2020-04-30 13:58:08 [INFO] Running synthtool
2020-04-30 13:58:08 [INFO] ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/tmpfs/src/github/synthtool/synthtool/__init__.py", line 21, in <module>
    from synthtool import update_check
  File "/tmpfs/src/github/synthtool/synthtool/update_check.py", line 19, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'
2020-04-30 13:58:08 [ERROR] Synthesis failed
HEAD is now at b74c0ce chore: regenerate README (#232)
Switched to branch 'autosynth'
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 576, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 457, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 566, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 352, in synthesize_loop
    synthesize_inner_loop(toolbox, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 362, in synthesize_inner_loop
    synthesizer, len(toolbox.versions) - 1
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 259, in synthesize_version_in_new_branch
    synthesizer.synthesize(self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 115, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.