- As of July 31, 2024, you can't access Dataflow SQL in the Google Cloud console.
- As of January 31, 2025, you can't use Dataflow SQL in the Google Cloud CLI.
Replacement: Beam SQL
Beam SQL lets you
query bounded and unbounded PCollections with SQL statements. If you're
working in an Apache Beam notebook, you can use the IPython
custom magic
beam_sql to speed up your pipeline development.
Beam SQL provides similar functionality as Dataflow SQL,
using SQL statements to query PCollections.
Migration
We recommend that you begin migrating your Dataflow SQL query to Beam SQL as soon as possible.
Moving to Beam SQL is a breaking change that affects any Dataflow SQL job or query. You will need to rebuild these jobs and queries.
With this change, you can't create new Dataflow SQL jobs. However, this change doesn't affect Dataflow SQL jobs that are already running.
For information about creating queries in Beam SQL, see
Beam SQL and beam_sql magic
and
Beam SQL overview.