<h1>RedShift</h1>
Relational database that can store up to 16PB of data and built to use BI Business Intelligence tools. Not standard and does not take the place of RDS.
What is big data?
3 V's
Volume, Variety, Velocity
The amount of data ranges from terabytes to petabytes of data. It can be in any format and come from any device, and you need to collect, analyze and process the data in a short time period.
<h1>EMR</h1>
ETL -Extract Transform Load
We take the large amounts of data from the source, Turn it into something usable, and ship it off to be used.
EMR - Elastic MapReduce
EMR is a AWS's ETL tool. A managed big data platform that allows one to process vasts amounts of data using open-source tools, such as Spark, Hive, HBase, Flink, Hudi, and Presto.
EMR lives inside a VPC and will spin up ec2 instances that are needed. So consider that. Where they may live in the VPC, how muh they cost, etc. It is a open-source cluster or a managed fleet of ec2 instances running open-source tools. like elastic-cache service.
EC2 Rules apply -you can use Rls and spot instances to reduce cost
<h1>Kinesis</h1>
Allows you to ingest, process, and analyze real-time streaming data. Like a highway, all about moving the data.
Data Streams
<ul>
<li>real time</li>
<li>unlimited in where you can send data to</li>
<li>You have to figure out how to consume data</li>
<li>does not auto scale</li>
</ul>
A producer is anything that is creating data. e.g. mobile device, ec2 instance, IoT device, Traditional server.
Take the info from the producer and feed into Kinesis Data Stream. Info is organized by shards.
Consumers take the data and deal with it. Direct it to the storage endpoint. e.g. S3, DynamoDb, Redshift, EMR, Kinesis Firehose.
Firehose
<ul>
<li>Data Transfer tool to get information to S3, Redshift, Elasticsearch, or Splunk.</li>
<li>near real time (within 60 sec)</li>
<li>Is plug and play with AWS</li>
</ul>
Firehose handles the scaling for you. Handles building the consumer. Has support for some third party end points
Kinesis Data Analytics
<ul>
<li>use standard sql, in real time as data is pushed through</li>
<li>easy</li>
<li>no servers</li>
<li>low cost</li>
</ul>
SQS does not offer real time message delivery, Kensis does
real time - Kinesis data streams
near real time - Kinesis Firehose
<h1>Athena</h1>
an inter active query service that makes it easy to analyze data in S3 using SQL. Talk to data without loading it into a database.
Glue - allows one to perform ETL workloads without managing underlying servers (EC2). It is a serverless data integration service that makes it easy to discover, prepare, and combine data. EMR without the BI tools.
You can use Athena to query logs stored in a s3 bucket.
Athena is serverless SQL
Glue is serverless ETL
<h1>QuickSight</h1>
Fully managed BI data visualization service. It allows one to easily create dashboards and share them within your company.
<h1>ElasticSearch</h1>

Analyze data in Amazon S3 using standard SQL

big-d Quiz

1 of 5 Score: 0

What is a valid use case for Amazon EMR?