Ever wanted to define alerts and monitor the status of your Cloud Dataflow jobs programmatically instead of checking some UI every 30 minutes? You can… Read More »How to programmatically monitor your Cloud Dataflow jobs
Ever wanted to track your resource usage and costs by specific Cloud Dataflow jobs? Cloud Dataflow recently started labeling billing records with Job Ids. Here… Read More »Calculating per-job Cloud Dataflow costs - now possible with job labels
What happens if you take a huge cross-section of the world’s news (The GDELT Project), mix it with the biggest online discussion website, and try… Read More »Predicting social engagement for the world’s news with TensorFlow and Cloud Dataflow: Part 1
I published an architecture of a serverless ETL solution for Sirocco on the GCP Big Data blog. This solution scales from a few news articles… Read More »Serverless ETL for Sirocco on Google Cloud