SAA-C03 ยท Data ingestion, lakes and analytics
AWS Glue crawlers, Data Catalog and columnar formats
Glue crawlers and the Data Catalog, Glue ETL jobs and converting CSV or JSON to columnar Parquet or ORC for faster, cheaper queries.
Key points
- Glue crawler: reads a store such as an S3 prefix, classifiers infer format and schema, writes table definitions (metadata only, never data) to the Glue Data Catalog.
- Runs on a schedule; Hive-style folders become partitions.
- Glue ETL jobs: catalog tables as sources and targets, e.g. CSV or JSON to Parquet or ORC.
- Columnar files: Athena reads only needed columns, they compress well, faster and cheaper.
Source: https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html
Practice questions
What does an AWS Glue crawler produce when it finishes a run over an S3 prefix?
- Table definitions in the AWS Glue Data Catalog
- A copy of the files converted to Parquet
- A new S3 bucket with the data sorted by date
- An Athena query result stored as CSV
Show answer
Answer: A. Table definitions in the AWS Glue Data Catalog
A crawler infers the schema and creates or updates table definitions and partitions in the Glue Data Catalog. It does not move, copy or convert the data files themselves; that is the job of Glue ETL jobs.
Why do columnar formats such as Parquet or ORC make analytics on S3 cheaper than CSV?
- A query can read only the columns it references
- The files stay readable in any plain text editor
- Values stored by column compress efficiently
- Athena cannot query CSV files at all
- They remove the need for a Data Catalog table
Show answer
Answer: A. A query can read only the columns it references, C. Values stored by column compress efficiently
A columnar file stores each column together, so an engine reads only the columns a query names, and similar values in a column compress well. Parquet is binary, not readable in a text editor, and Athena can query CSV too, so conversion is an optimization, not a requirement.
An AWS Glue crawler copies the crawled files into the Data Catalog so that queries read them from there.
Show answer
Answer: False
False. The Data Catalog stores only metadata: table names, schemas, partitions and the S3 location. The data stays where it is, and engines such as Athena read it from S3 using that metadata.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
