Skip to content

Refactor write client code to use the generated instead of the handwritten BigQuery client #1249

@stephaniewang526

Description

@stephaniewang526

The fact that google-cloud-bigquerystorage Write client (v1beta2) is using google-cloud-bigquery as a dependency means it would introduce a cyclic dependency when we integrate it into google-cloud-bigquery for high throughput Write in the future. This is also currently blocking us from using google-cloud-bigquerystorage client in google-cloud-bigquery for the Read API features (v1). We have found a temporary workaround to unblock ourselves on the Read side but this refactoring is necessary before we GA this client.

Specifically, 3 classes require full refactoring:

  1. /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:
  2. /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:
  3. /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/JsonStreamWriter.java:

Full trace:

 Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project google-cloud-bigquerystorage: Compilation failure: Compilation failure: 
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[18,33] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[19,33] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[20,33] cannot find symbol
[ERROR]   symbol:   class StandardSQLTypeName
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[25,36] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[31,31] cannot find symbol
[ERROR]   symbol:   class StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[53,48] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[67,53] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[18,33] cannot find symbol
[ERROR]   symbol:   class BigQuery
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[19,33] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[20,33] cannot find symbol
[ERROR]   symbol:   class LegacySQLTypeName
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[21,33] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[22,33] cannot find symbol
[ERROR]   symbol:   class Table
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[23,33] cannot find symbol
[ERROR]   symbol:   class TableId
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[24,41] package com.google.cloud.bigquery.testing does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[46,11] cannot find symbol
[ERROR]   symbol:   class BigQuery
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[74,31] cannot find symbol
[ERROR]   symbol:   class BigQuery
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[99,49] cannot find symbol
[ERROR]   symbol:   class BigQuery
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[104,11] cannot find symbol
[ERROR]   symbol:   class TableId
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[257,47] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[310,7] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[409,7] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/JsonStreamWriter.java:[22,33] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: package com.google.cloud.bigquery
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/JsonStreamWriter.java:[208,62] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.JsonStreamWriter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[27,16] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[28,16] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[29,16] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[44,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[43,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[42,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[41,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[40,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[39,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[38,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[37,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[36,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[35,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[34,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[33,16] cannot find symbol
[ERROR]   symbol:   variable StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[32,32] cannot find symbol
[ERROR]   symbol:   class StandardSQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.BQV2ToBQStorageConverter
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/BQV2ToBQStorageConverter.java:[70,46] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[86,7] cannot find symbol
[ERROR]   symbol:   class RemoteBigQueryHelper
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[86,45] cannot find symbol
[ERROR]   symbol:   variable RemoteBigQueryHelper
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[109,12] cannot find symbol
[ERROR]   symbol:   variable TableId
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[264,12] cannot find symbol
[ERROR]   symbol:   variable REPEATED
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[274,12] cannot find symbol
[ERROR]   symbol:   variable REQUIRED
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[284,12] cannot find symbol
[ERROR]   symbol:   variable NULLABLE
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[318,5] cannot find symbol
[ERROR]   symbol:   class LegacySQLTypeName
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[365,13] cannot find symbol
[ERROR]   symbol:   variable Schema
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[421,10] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[442,10] cannot find symbol
[ERROR]   symbol:   class Field
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[450,59] package Field does not exist
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[512,5] cannot find symbol
[ERROR]   symbol:   class TableId
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[513,5] cannot find symbol
[ERROR]   symbol:   class Table
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] /usr/local/google/home/stephwang/Desktop/stephdev/java-bigquerystorage/google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1beta2/SchemaCompatibility.java:[514,5] cannot find symbol
[ERROR]   symbol:   class Schema
[ERROR]   location: class com.google.cloud.bigquery.storage.v1beta2.SchemaCompatibility
[ERROR] -> [Help 1]

Metadata

Metadata

Labels

api: bigquerystorageIssues related to the googleapis/java-bigquerystorage API.type: cleanupAn internal cleanup or hygiene concern.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions