Create streams and ingest data

The first step in analyzing your video data with an application is creating a pipeline for the continuous flow of data. Streams offers the ability to ingest real-time video data, which then can be used as input for models or stored in a Warehouse.

Create a Stream

To be able to stream video data you must first create a stream.

The first time you create a stream in a new project it can take some time to create the stream. This time is around 30 minutes. This time is due to the system needing to initialize a computation cluster for subsequent computations.

Console

Create a stream in the Google Cloud console.

  1. Open the Streams tab of the Vertex AI Vision dashboard.

    Go to the Streams tab

  2. Click Register.

  3. Enter the stream name and select a region. You can click Add Row to register multiple streams at the same time.

  4. Click the Register button to create one or more streams.

Ingest videos

After you have created a stream you can begin to ingest data using that stream.

Some limitations and considerations apply to ingested video:

  • The input video source must have the following specifications:

    If the video doesn't meet these specifications Vertex AI Vision may not process the input well.

  • Audio is dropped during ingestion.
  • If the stream is part of an app connected to a warehouse, video parameters (such as FPS or resolution) must be the same for the whole stream ; variations in RTSP video data parameters or local video data parameters are not supported.
  • Due to model startup latency, ingested videos may have the first few seconds of content missing. This amount of time can reach up to fifteen seconds.
  • Ingestion termination can happen after long connection (~5 hours, on average). There's no reconnection capability from the vaictl tool ; users must manually reconnect.
  • While vaictl is a useful tool for ingesting video data, the tool does not automatically handle recovery from network errors. These errors may either come from the data-source side or the Cloud-ingestion side. It's the user's responsibility to create a restart script to handle unexpected vaictl operation terminations.

Vertex AI Vision SDK

To send a request to ingest video data using an existing stream you must install the Vertex AI Vision SDK.

Make the following variable substitutions:

  • PROJECT_ID: Your Google Cloud project ID.
  • LOCATION_ID: Your location ID. For example, us-central1. More information. Supported regions.
  • LOCAL_FILE.EXT: The filename of a local video file. For example, my-video.mp4.
  • STREAM_ID: The stream ID that you created in the cluster For example, input-1.
  • RTSP_ADDRESS: The address of your Real Time Streaming Protocol (RTSP) feed. For example, rtsp://my-ip-camera.

Local video data:

# This command streams a video file to a stream. Streaming ends when the video ends.
vaictl -p PROJECT_ID \
         -l LOCATION_ID \
         -c application-cluster-0 \
         --service-endpoint visionai.googleapis.com \
send video-file to streams STREAM_ID --file-path LOCAL_FILE.EXT

Local video data (looped):

# This command streams a video file to a stream. Video is looped into the stream until you stop the command.
vaictl -p PROJECT_ID \
         -l LOCATION_ID \
         -c application-cluster-0 \
         --service-endpoint visionai.googleapis.com \
send video-file to streams STREAM_ID --file-path LOCAL_FILE.EXT --loop

Real Time Streaming Protocol (RTSP) feed data:

# This command will send an RTSP feed into the stream.
# This command has to run in the network that has direct access to the RTSP feed.
vaictl -p PROJECT_ID \
         -l LOCATION_ID \
         -c application-cluster-0 \
         --service-endpoint visionai.googleapis.com \
send rtsp to streams STREAM_ID --rtsp-uri RTSP_ADDRESS

Play ingested videos

After you have created a stream and sent video content to the stream you can play the video content.

If you leave the stream view tab open for more than one hour, you must refresh the page to continue to load video contents.

Console

Play ingested video in the Google Cloud console.

  1. Open the Streams tab of the Vertex AI Vision dashboard.

    Go to the Streams tab

  2. Click on the stream name from the streams list.

  3. In the video stream detail page the ingested video live view automatically loads.