#Metabase aws athena how to

We use an Amazon CloudWatch Events rule with the following event pattern to trigger an AWS Lambda function for every Athena query run:ĬREATE EXTERNAL TABLE `athena_query_details`(.The solution consists of the following high-level steps: The following diagram illustrates our solution architecture.

#Metabase aws athena how to

We also demonstrate how to visualize audit data collected for a few key metrics (data scanned per workgroup and data scanned per user) using Amazon QuickSight. In this post, we provide a solution that collects detailed statistics of every query run in Athena and stores it in your data lake for auditing. Repetitive queries run by individual users.

Amount of data scanned by different workgroups.

Amount of data scanned by individual users.

As a result, it’s desirable to have metrics that provide the following details: In such cases, it’s beneficial to monitor Athena usage to ensure cost-effectiveness, avoid any performance bottlenecks, and adhere to security best practices. Typically, multiple users within an organization operate under different Athena workgroups and query various datasets in Amazon S3. Cancelled queries are charged based on the amount of data scanned. You’re not charged for data definition language (DDL) statements like CREATE, ALTER, or DROP TABLE, statements for managing partitions, or failed queries.

You pay only for the queries that you run and the charges are based on the amount of data scanned by each query. Athena scales automatically-running queries in parallel-so results are fast, even with large datasets and complex queries. It’s a serverless platform with no need to set up or manage infrastructure. Amazon Athena is an interactive query service that makes it easy to analyze data directly in Amazon Simple Storage Service (Amazon S3) using standard SQL.