This repository contains the code and resources for working with Uber dataset for data analytics on Google Cloud Platform.
The project involves various stages including data storage, ETL (Extract, Transform, Load) processes, analytics using BigQuery, and building dashboards with Looker.
- Contains quantitative measures or metrics that are used for analysis
- Typically contains foreign keys that link to dimension tables
- Contains columns that have high cardinality and change frequently
- Contains columns that are not useful for analysis by themselves, but are necessary for calculating metrics
- Contains columns that describe attributes of the data being analyzed
- Typically contains primary keys that link to fact tables
- Contains columns that have low cardinality and don't change frequently
- Contains columns that can be used for grouping or filtering data for analysis
- Data Storage through Cloud: Utilizing Google Cloud Storage for storing the dataset.
- ETL: Employing MergeMV and Compute Engine for ETL processes. Marge: An open source data pipeline tool for transforming and integrating data. It is like Airflow.
- Analytics/BigQuery: Performing analytics using BigQuery for insights.
- Looker: Building dashboards for visualization and exploration.
-
Programming Language: Python
-
Google Cloud Platform Services:
- Google Storage
- Compute Engine
- BigQuery
- Looker Studio
-
Dataset: The dataset used in this project is available here.
-
Data Dictionary: The data Dictionary of dataset used used in this project here.
-
Addictional Data: For further information, refer to the NYC TLC Trip Record Data and its Data Dictionary.
To understand the Google Cloud Platform services utilized in this project, refer to the following:
- Cloud Storage
- Compute Engine
- BigQuery
- Looker
For any inquiries or feedback, feel free to reach out to ericmaniraguha@gmail.com.