9f5e6791db
Funciones CRUD completas para BigQuery: auth, datasets, tables, queries, jobs, routines, load/export. Tipo BQClient como wrapper del SDK oficial.
2.6 KiB
2.6 KiB
name, kind, lang, domain, version, purity, signature, description, tags, uses_functions, uses_types, returns, returns_optional, error_type, imports, params, output, tested, tests, test_file_path, file_path
| name | kind | lang | domain | version | purity | signature | description | tags | uses_functions | uses_types | returns | returns_optional | error_type | imports | params | output | tested | tests | test_file_path | file_path | ||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bq_load_from_file | function | py | infra | 1.0.0 | impure | def bq_load_from_file(client: BQClient, file_path: str, dataset_id: str, table_id: str, source_format: str = 'CSV', write_disposition: str = 'WRITE_APPEND', autodetect: bool = True, skip_leading_rows: int = 0) -> dict | Carga datos desde un archivo local a una tabla BigQuery usando load_table_from_file del SDK. Equivalente a bq_load_from_gcs pero para disco local. |
|
false | error_go_core |
|
|
dict con {job_id: ID del LoadJob, rows_loaded: filas cargadas, status: DONE o FAILED} | false | python/functions/bigquery/queries.py |
Ejemplo
from bigquery.client import bq_auth
from bigquery.queries import bq_load_from_file
client = bq_auth("my-project")
# Cargar CSV local con cabecera
result = bq_load_from_file(
client,
"/tmp/export_users.csv",
"my_dataset", "users",
skip_leading_rows=1,
write_disposition="WRITE_TRUNCATE",
)
print(f"Cargadas {result['rows_loaded']} filas — job: {result['job_id']}")
# Cargar JSONL local
result = bq_load_from_file(
client,
"/data/events.jsonl",
"my_dataset", "events",
source_format="NEWLINE_DELIMITED_JSON",
)
Notas
El archivo se abre en modo binario (rb) y se sube directamente al job de BigQuery.
Para archivos muy grandes, preferir bq_load_from_gcs — subir primero a GCS y luego
cargar desde ahi es mas eficiente y permite paralelismo.
La funcion bloquea hasta que el job termina (job.result()). Los archivos Parquet y
Avro no admiten skip_leading_rows — ese parametro solo aplica para CSV.