Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dbapi: Skip storage client fetch when results cached #1745

Closed
r1b opened this issue Dec 6, 2023 · 3 comments · Fixed by #1747
Closed

dbapi: Skip storage client fetch when results cached #1745

r1b opened this issue Dec 6, 2023 · 3 comments · Fixed by #1747
Assignees
@tswast
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@r1b
Copy link
Contributor

r1b commented Dec 6, 2023

Is your feature request related to a problem? Please describe.
This is the same problem as #589, but when using the DBAPI interface

Describe the solution you'd like
We'd like the DBAPI interface to include the result fetching optimization added in #1723

This is mentioned in #362:

Update DB-API to not call BQ Storage API if cached results are the only page

but wanted to be sure it was tracked :)

Describe alternatives you've considered
N/A (we use BigQuery via SQLAlchemy, so we cannot use an alternative interface)

Additional context
N/A
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Dec 6, 2023
@tswast tswast self-assigned this Dec 6, 2023
@tswast tswast added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Dec 6, 2023
@tswast
Copy link
Contributor

tswast commented Dec 6, 2023

Thanks for the reminder! I'll work on this, based on the work I'm doing in #1722 which optimized further.

This was referenced Dec 8, 2023
Merged
Closed
@tswast
Copy link
Contributor

tswast commented Dec 12, 2023

I did some benchmarking with my change #1747 (comment) which is currently out for review. The summary is:

Small query results (e.g. SELECT 1) have a (1.18 / 0.319) = 3.7x speedup! For medium-sized results/queries, this optimization has less dramatic results at (1.67 / 1.63) = 1.2x speedup and not statistically significant.

gcf-merge-on-green bot pushed a commit that referenced this issue Dec 19, 2023
@tswast
perf: DB-API uses more efficient query_and_wait when no job ID is p…
d225a94 
…rovided (#1747)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://togithub.com/googleapis/python-bigquery/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #1745  🦕
@nickmarx12345678
Copy link
Contributor

@tswast thanks for knocking this out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tswast tswast

Labels
api: bigquery Issues related to the googleapis/python-bigquery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Milestone
No milestone
3 participants
@tswast @nickmarx12345678 @r1b