diff options
Diffstat (limited to 'src')
49 files changed, 1196 insertions, 0 deletions
diff --git a/src/README.md b/src/README.md new file mode 100644 index 0000000..9f679e5 --- /dev/null +++ b/src/README.md @@ -0,0 +1,7 @@ +# Introduction + +Welcome to my wiki, this is where I document my learning, including notes on certifications and courses I take, programs I use and workflows I re-use. + +I believe in creating a central repository for your knowledge, instead of relying on external services to consume tidbits of information. By collating and documenting what I learn, it helps my synthesise information and have persistent reference points written and maintained by me. + +It is built using [mdBook](https://rust-lang.github.io/mdBook/), which is a really great CLI for generating books/wikis from markdown files. diff --git a/src/SUMMARY.md b/src/SUMMARY.md new file mode 100644 index 0000000..fad51c1 --- /dev/null +++ b/src/SUMMARY.md @@ -0,0 +1,26 @@ +# Summary + +[Introduction](README.md) + +# Certifications + +- [AWS Solutions Architect](./certifications/aws-solutions-architect/README.md) + - [intro](./certifications/aws-solutions-architect/intro.md) + - [infra](./certifications/aws-solutions-architect/infra.md) + - [iam](./certifications/aws-solutions-architect/iam.md) + - [cli-sdk](./certifications/aws-solutions-architect/cli-sdk.md) + - [ec2](./certifications/aws-solutions-architect/ec2.md) + - [ips](./certifications/aws-solutions-architect/ips.md) + - [placement groups](./certifications/aws-solutions-architect/ec2-placement-groups.md) + - [route 53](./certifications/aws-solutions-architect/route-53.md) + - [eni](./certifications/aws-solutions-architect/eni.md) + - [ec2 storage](./certifications/aws-solutions-architect/ec2-storage.md) + - [ami](./certifications/aws-solutions-architect/ami.md) + - [ec2 instance store](./certifications/aws-solutions-architect/ec2-instance-store.md) + + + +<!-- # Programs --> +<!----> +<!-- - [Taskwarrior](./programs/taskwarrior/README.MD) --> +<!-- - [Installation](./programs/taskwarrior/1.installation.md) --> diff --git a/src/certifications/aws-solutions-architect/RDS.md b/src/certifications/aws-solutions-architect/RDS.md new file mode 100644 index 0000000..e1c95cf --- /dev/null +++ b/src/certifications/aws-solutions-architect/RDS.md @@ -0,0 +1,204 @@ +# Relational Database Services (RDS) + +- RDS is a managed database service from AWS +- It uses SQL and supports the following databases: + +1. Postgres +2. MySQL +3. MariaDB +4. Oracle +5. Microsoft SQL server +6. IBM DB2 +7. Aurora (AWS proprietary service) + +## Why use a managed service? + +AWS manages the whole service for you, including deployment of databases onto infrastructure. Because it's managed, you don't have access to the underlying infra, so you can't SSH into the DB servers. + +Advantages of this: + +- Automated provisioning, OS patches +- Backups +- Monitoring dashboards +- Scaling (horizontal and vertical) +- Multi AZ (recovery) +- Read replicas (performance) + +## Storage Auto Scaling + +AWS can scale DB stores automatically depending on usage. For e.g when you are running out of DB space. + +You have to set a **Maximum Storage Threshold**. + +> Auto scaling is good for apps with unpredictable DB operations## Storage Auto Scaling + +## Read Replicas VS Multi AZ + +### Read Replicas + +Read replicas are DB replicas of the main DB that allow for more read operations (scalability). + +They work by replicating the main DB and then allowing read operations to those replicas. + +The data between the main DB and replica DBs is *eventually consistent*, meaning that they will eventually have identical data, but there is a chance a read operation to a replica will receive back outdated data. + +Replicas can also be promoted to their own DB. + + + + +#### Network Costs for replicas + +There are network costs associated with replicas. If you replicate across a different AZ, a fee is incurred, else it's free. + +### Multi AZ + +Multi AZ RDS are DB instances that are on standby should something go wrong with the master DB. + +This helps increase availability + +> Multi AZ RDS is good for database availability + +There is just one DNS name needed and if there is network loss, instance of storage failure, read and write operations will be passed to the instance on standby: + + + +**There is no downtime associated when creating Multi AZ - you just modify the DB** + +> Important to know that you can also use read replicas for Disaster Recovery (DS) + +## RDS Custom + +While it's been mentioned that RDS is a fully managed service, with two databases you do get OS access and can SSH into the instances: + +- Orcale +- Microsoft SQL server + +For these two DBs you can: + +- Configure settings +- Install patchesEnable features +- Access EC2 instance using SSH + +> RDS is a managed services, except for Oracle and MS SQL, which allow customisation and EC2 access + +## Amazon Aurora + +This is Amazon's proprietary database offering, it's not open source. It: + +- Aurora works with bothPostgres and MySql. +- Has 5x better perf than MySQL and 3x the perf of Postgres +- Storage automatically grows in 10GB increments, up to 128TB +- Can have 15 read replicas +- Failover is instantaneous + +### Aurora availability and read scaling + +Aurora created 6 copies of data across 3 AZs (diagram). + +The storage is self healing and auto expanding. + +One instance takes writes (the master) and the data is replicated across the instances, which can be used for read operations. + + + + +## Aurora Cluster + +Aurora has a: + +1. Shared volume +2. 1 master writer - one endpoint +3. 5 readers - one endpoint that scale automatically + + + +## Advanced - Aurora replicas and auto scaling + +If read endpoints receive much more traffic then the read replicas will autoscale: + + + +## Avanced - replicas and custom endpoints + +By default, replicas share the same endpoint, but you can specify a custom endpoint should some of your replicas be unique, for example the DBs are larger or more powerful: + + + +## Advanced - Aurora serverless + +Serverless option from AWS that: + +- automatically scales depending on usage +- no storage planning needed +- pay per second pricing model + +## Advanced - global availability + +For global availability you can: + +1. Create Aurora cross region availability using read replicas +2. Use an Aurora global database (recommended): + +The global database has: + +- 1 primary region +- up to 5 additional read only regions +- 16 read replicas in the regions +- helps decrease latency globally +- replication takes less than 1s + +> In the exam when "cross region replication < 1s" is mentioned, it's referring to Aurora Global + +## Advanced - Aurora Machine Learning + +SQL integration with other AWS ML tools: SageMaker and Comprehend. It basically takes data from your Aurora tables and uses them to power ML tools. The data will be routed through Aurora: + + + +## RDS Backups + +There are two ways to backup: + +1. Automated Backups: daily full backups, can restore from any point up to 5 min before backup (so any time in the past up until 5min ago). 1-35 days retention +2. Manual DB Snapshots: manually triggered and retention as long as you want + +> To save money, instead of stopping a DB (you will still pay) you can create a snapshot and restore from there in the future + +## Aurora backups + +It's similar, with automated backups and manual DB snapshots. Automated backups cannot be turned off! + +## Restoring operations + +Two options: + +1. Restoring automatically from backup/snapshot. This will create a new DB. +2. Restore MYSQL / Aurora from S3. Involves creating a backup on S3 and then restoring from there. For Aurora you have to use Percona XtraBackup for this. + +## Aurora DB cloning + +Creates new Aurora cluster from an existing one. You can do this to run DB testing, for example. + +When the new cluster is created and new write operations are made to it, new storage is allocated only to the new DB cluster. + +## RDS Proxy + +There is a fully managed proxy service with RDS. A proxy is a middle man between client applications and (in this case) a DB server. + +Proxies help reduce load on DB servers by improving the performance, scalability and security of DB communication. They do this by: load balancing, pooling connections, caching, query optimisation etc. + +The RDS proxy service: + +- Allows apps to pool and share DB connections +- Improves DB efficiency by reducing stress on DB resources +- Autoscales, multi AZ +- Supports SQL (Postgres, mySQL, MariaDB etc) +- Enforce IAM auth and store creds in AWS secrets manager +- Proxy is never publicly acessible + + + + + + diff --git a/src/certifications/aws-solutions-architect/README.md b/src/certifications/aws-solutions-architect/README.md new file mode 100644 index 0000000..44e4abe --- /dev/null +++ b/src/certifications/aws-solutions-architect/README.md @@ -0,0 +1 @@ +# AWS Solutions Architect diff --git a/src/certifications/aws-solutions-architect/ami.md b/src/certifications/aws-solutions-architect/ami.md new file mode 100644 index 0000000..e8600b3 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ami.md @@ -0,0 +1,18 @@ +# Amazon Machine Image (AMI) + +AMIs are customisations of an EC2 instance. + +It's basically a snapshot of an instance that can be used to launch new, customised EC2 instances. This helps with faster boot times. + +So far we have been launching using Amazon's own Linux AMIs. But we can: + +- Create our own AMIs +- Used 3rd party vendor AMIs (can cost) + +This allows for the following: + +- Create an instance in one AZ +- Create AMI from that instance +- Launch a new instance from that AMI in another AZ + + diff --git a/src/certifications/aws-solutions-architect/assets/2025-02-10-19-51-19.png b/src/certifications/aws-solutions-architect/assets/2025-02-10-19-51-19.png Binary files differnew file mode 100644 index 0000000..118c29c --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/2025-02-10-19-51-19.png diff --git a/src/certifications/aws-solutions-architect/assets/alb-query.png b/src/certifications/aws-solutions-architect/assets/alb-query.png Binary files differnew file mode 100644 index 0000000..ef9631d --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/alb-query.png diff --git a/src/certifications/aws-solutions-architect/assets/alb.png b/src/certifications/aws-solutions-architect/assets/alb.png Binary files differnew file mode 100644 index 0000000..d7236e6 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/alb.png diff --git a/src/certifications/aws-solutions-architect/assets/aurora-cluster.png b/src/certifications/aws-solutions-architect/assets/aurora-cluster.png Binary files differnew file mode 100644 index 0000000..0f436a7 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/aurora-cluster.png diff --git a/src/certifications/aws-solutions-architect/assets/aurora-custom-endpoint.png b/src/certifications/aws-solutions-architect/assets/aurora-custom-endpoint.png Binary files differnew file mode 100644 index 0000000..5203d5d --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/aurora-custom-endpoint.png diff --git a/src/certifications/aws-solutions-architect/assets/aurora-ml.png b/src/certifications/aws-solutions-architect/assets/aurora-ml.png Binary files differnew file mode 100644 index 0000000..2d08d5b --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/aurora-ml.png diff --git a/src/certifications/aws-solutions-architect/assets/aurora-read-scaling.png b/src/certifications/aws-solutions-architect/assets/aurora-read-scaling.png Binary files differnew file mode 100644 index 0000000..33bff97 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/aurora-read-scaling.png diff --git a/src/certifications/aws-solutions-architect/assets/aurora-scaling.png b/src/certifications/aws-solutions-architect/assets/aurora-scaling.png Binary files differnew file mode 100644 index 0000000..37beefd --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/aurora-scaling.png diff --git a/src/certifications/aws-solutions-architect/assets/calculated-checks.png b/src/certifications/aws-solutions-architect/assets/calculated-checks.png Binary files differnew file mode 100644 index 0000000..05edc9d --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/calculated-checks.png diff --git a/src/certifications/aws-solutions-architect/assets/czlb.png b/src/certifications/aws-solutions-architect/assets/czlb.png Binary files differnew file mode 100644 index 0000000..098cf2d --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/czlb.png diff --git a/src/certifications/aws-solutions-architect/assets/dns.png b/src/certifications/aws-solutions-architect/assets/dns.png Binary files differnew file mode 100644 index 0000000..cec34ca --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/dns.png diff --git a/src/certifications/aws-solutions-architect/assets/elasticache.png b/src/certifications/aws-solutions-architect/assets/elasticache.png Binary files differnew file mode 100644 index 0000000..0c83acd --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/elasticache.png diff --git a/src/certifications/aws-solutions-architect/assets/glb-2.png b/src/certifications/aws-solutions-architect/assets/glb-2.png Binary files differnew file mode 100644 index 0000000..7aad11e --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/glb-2.png diff --git a/src/certifications/aws-solutions-architect/assets/glb.png b/src/certifications/aws-solutions-architect/assets/glb.png Binary files differnew file mode 100644 index 0000000..e7365fd --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/glb.png diff --git a/src/certifications/aws-solutions-architect/assets/hat.png b/src/certifications/aws-solutions-architect/assets/hat.png Binary files differnew file mode 100644 index 0000000..d6fbe9c --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/hat.png diff --git a/src/certifications/aws-solutions-architect/assets/health-check.png b/src/certifications/aws-solutions-architect/assets/health-check.png Binary files differnew file mode 100644 index 0000000..c57f78a --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/health-check.png diff --git a/src/certifications/aws-solutions-architect/assets/hosted-zones.png b/src/certifications/aws-solutions-architect/assets/hosted-zones.png Binary files differnew file mode 100644 index 0000000..e2d40db --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/hosted-zones.png diff --git a/src/certifications/aws-solutions-architect/assets/loadbalancer.png b/src/certifications/aws-solutions-architect/assets/loadbalancer.png Binary files differnew file mode 100644 index 0000000..d732ba5 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/loadbalancer.png diff --git a/src/certifications/aws-solutions-architect/assets/moss.png b/src/certifications/aws-solutions-architect/assets/moss.png Binary files differnew file mode 100644 index 0000000..c299827 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/moss.png diff --git a/src/certifications/aws-solutions-architect/assets/multi-az-rds.png b/src/certifications/aws-solutions-architect/assets/multi-az-rds.png Binary files differnew file mode 100644 index 0000000..bffe7ae --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/multi-az-rds.png diff --git a/src/certifications/aws-solutions-architect/assets/nlb.png b/src/certifications/aws-solutions-architect/assets/nlb.png Binary files differnew file mode 100644 index 0000000..8f733cf --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/nlb.png diff --git a/src/certifications/aws-solutions-architect/assets/private-health-checks.png b/src/certifications/aws-solutions-architect/assets/private-health-checks.png Binary files differnew file mode 100644 index 0000000..a2e50f8 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/private-health-checks.png diff --git a/src/certifications/aws-solutions-architect/assets/rds-replicas.png b/src/certifications/aws-solutions-architect/assets/rds-replicas.png Binary files differnew file mode 100644 index 0000000..ec98cae --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/rds-replicas.png diff --git a/src/certifications/aws-solutions-architect/assets/redis-sorted-sets.png b/src/certifications/aws-solutions-architect/assets/redis-sorted-sets.png Binary files differnew file mode 100644 index 0000000..85a9995 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/redis-sorted-sets.png diff --git a/src/certifications/aws-solutions-architect/assets/session-store.png b/src/certifications/aws-solutions-architect/assets/session-store.png Binary files differnew file mode 100644 index 0000000..3c0fe29 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/session-store.png diff --git a/src/certifications/aws-solutions-architect/assets/sni.png b/src/certifications/aws-solutions-architect/assets/sni.png Binary files differnew file mode 100644 index 0000000..a7f2ed6 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/sni.png diff --git a/src/certifications/aws-solutions-architect/assets/ssl.png b/src/certifications/aws-solutions-architect/assets/ssl.png Binary files differnew file mode 100644 index 0000000..023367e --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/ssl.png diff --git a/src/certifications/aws-solutions-architect/assets/ttl.png b/src/certifications/aws-solutions-architect/assets/ttl.png Binary files differnew file mode 100644 index 0000000..3378124 --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/ttl.png diff --git a/src/certifications/aws-solutions-architect/assets/urlstructure.png b/src/certifications/aws-solutions-architect/assets/urlstructure.png Binary files differnew file mode 100644 index 0000000..84a208b --- /dev/null +++ b/src/certifications/aws-solutions-architect/assets/urlstructure.png diff --git a/src/certifications/aws-solutions-architect/cli-sdk.md b/src/certifications/aws-solutions-architect/cli-sdk.md new file mode 100644 index 0000000..a75a570 --- /dev/null +++ b/src/certifications/aws-solutions-architect/cli-sdk.md @@ -0,0 +1,29 @@ +# AWS CLI, SDK and Cloud Shell + +There are 3 ways to access AWS: + +1. The web console +2. AWS CLI +3. AWS SDKs + +The CLI is open source and allows you to make operations from the command line. + +The SDKs allow you to access AWS programatically through API calls using a set of language specific APIs, including: JS, Node, PHP, Python, .NET, Ruby, Java, Go, C++. + +## AWS CLI + +You need to use an access token to setup the AWS CLI. You define access tokens per user and then user : + +`aws configure` to create access for that user from the command line. + +## Cloudshell + +You can use cloudshell in the AWS console as well. + +This will automatically use the active user's credentials. It allows for : + +- File downloads +- Uploads +- etc etc + +It is only available in some regions, however. diff --git a/src/certifications/aws-solutions-architect/ec2-instance-store.md b/src/certifications/aws-solutions-architect/ec2-instance-store.md new file mode 100644 index 0000000..4886424 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ec2-instance-store.md @@ -0,0 +1,8 @@ +Hardware disk attached to EC2 instance, *not* a network drive, like EBS Volumes. + +> EC2 instance stores are higher performance storage for EC2 instance that are attached to the instance and aren't network drives + +- Better I/O performenace +- EC2 instance termination will lose the disk (ephemeral) +- Not for long term storage, instead use EBS +- Backups are your responsibility diff --git a/src/certifications/aws-solutions-architect/ec2-placement-groups.md b/src/certifications/aws-solutions-architect/ec2-placement-groups.md new file mode 100644 index 0000000..b32c93f --- /dev/null +++ b/src/certifications/aws-solutions-architect/ec2-placement-groups.md @@ -0,0 +1,14 @@ +# EC2 Placement Groups + +Placement groups allow you to define where your EC2 instances are deployed on AWS infrastructure. + +A placement group is either: + +1. *A cluster*: puts your instances in a low latency group in the same AZ. This helps with networking as instances are close to one another. Drawback is if the AZ fails, they all fail. **use case**: Good for big data jobs, apps that need low latency between instances. +2. *Spread*: Think of this as opposite to clusters. Each instance is on different hardware across different AZs. This means reduced failure risk. Limited to 7 AZ per placement group. **Use case**: maximum high availability. +3. *Partition*: spreads instances across different partitions within an AZ. Each partition represents a rack in AWS. Instances are distributed across different hardware racks and AZs in same region. **use cases**: Big data application, which are petition aware. + + + + + diff --git a/src/certifications/aws-solutions-architect/ec2-storage.md b/src/certifications/aws-solutions-architect/ec2-storage.md new file mode 100644 index 0000000..33505d6 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ec2-storage.md @@ -0,0 +1,97 @@ +# EC2 storage options + +Storage options for EC2 instances + +## EBS Volume + +Elastic Block Store - network drive. Allows for data persistence. + +Can only be mounted to 1 instance at a time. It's bound to AZ. But you can attach more than one volume to an instance + +> Like a network USB stick. It's not a physical drive, it uses the network, so there is latency. + +You can move it, but you've gotta snapshot the volume. + +You can decide to terminate the EBS volume on instance termination + +## EBS Snapshot + +- You can snapshot your volumes to act as a backup. +- Recommended to detach volume +- Can copy across AZs - allowing volume restoration in another AZ +- Copying to archive tier is much cheaper, but restoration is longer +- Snapshot deletions can be recovered from a recycle bin +- Fast Snapshot Restore (FSR) helps with latency on restoration +- Can also create other volumes from existing snapshots, which can be in a different AZ + +## EBS Volume Types + +There are 6 types of EBS volume types + +- GP2 / GP3 (SSD): perf/price balance +- io1 / io2 Block Express (SSD): High perf for low latency and high throughput +- st 1 (HDD): Low cost, frequent access, throughput intensive workloads +- sc 1 (HDD): Lowest cost for less frequent workloads + +Characterised by SIZE | THROUGHPUT | IOPS (I/O Ops per sec) + +> Only GP2/GP3 and io1 / io2 can be used at boot volumes + +### General purpose SSD (gp2/gp3) + +- Cost effective, low latency +- System boots +- 1GiB - 16 TiB +- Gp3: 2k - 16k IOPS & 125 MiB/s - 1k MiB/s +- Gp2: IOPS up to 16k. volume size and IOPS are linked + +### Provisioned IOPS volumes (io1/1o2) + +- Critical business apps +- Apps that nmeed more than 16k IOPS +- Great for DB workloads +- io1: 4gb - 16tb max 64k IOPS +- io2: 4gb - 64gb - max 256k IOPS + +### HDD (st1/sc1) + +- Can't be boot volumes +- 125gb - 16tb +- Throughpout optimised (st 1) - good for big data +- Infrequent access (low cost) - sc 1 + +Best resource: https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volume-types.html + +### EBS multi attach + +For io1/2 families you can attach same EBS volume to multiple EC2 instances in the same AZ. + +This is good for high availability and concurrent writes. + +> Only good for one AZ and 16 EC2 instances at the same time + +## EBS Encryption + +- Data encrypted at rest +- In flight encryption +- Snapshot encryption +- Uses KMs (AES-256). + +## Elastic file system (EFS) + +Allows for a network file system that allows multiple AZ instances to connect to the EFS through a security group. + + + +Use cases: content management, web serving, data sharing, Wordpress. Only Linux compatible (POSIX) system and scales automatically. + +It has different perf modes: + +- General purpose +- Throughput mode for Max I/O + +And storage classes: + +- standard +- infrequent access +- archive diff --git a/src/certifications/aws-solutions-architect/ec2.md b/src/certifications/aws-solutions-architect/ec2.md new file mode 100644 index 0000000..257bb29 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ec2.md @@ -0,0 +1,122 @@ +# EC2 + +An EC2 instance is just a virtual machine that you hire. It stands for **Elastic Compute Cloud** and is infrastructure as a service. + +You can also: + +- Store data on virtual drives +- Distribute load across machines +- Scale services + +You can choose what you want your virtual machine to be and its power, including CPU, RAM, networking capabilities etc. You can also choose between Mac, Linux and Windows machines. + +## Bootstrapping + +What the machine does at launch can be controlled using bootstrapping scripts + +## EC2 instance types + +https://aws.amazon.com/ec2/instance-types/ +https://instances.vantage.sh/ +There are different types of EC2 instances, designed for different purposes. You can find them above. + +Using _m5.2xlarge_ as an example, the naming convention is: + +- **m**: instance class, in this case m means general purpose +- **5**: generation (AWS improves gens over time) +- **2xlarge**: size, so the CPU and processing power + +### General purpose + +### Compute optimized + +High performance with good CPU. Examples are of the **C** name + +### Memory optimized + +High RAM. High performance for databases, cache stores, big unstructured data. An example are the **R** instances + +### Storage optimised + +Good for high, sequential read and write access to large data sets. + +Examples: + +- Databases +- Cache for in memory dbs +- Online transactioning systems +- Distributed filed systems + +## EC2 instance firewalls + +You can control who can access the EC2 instance and how your EC2 instance interacts with the internet using security groups. + +### Security groups + +Security groups contain _allow_ rules that can reference IPs or groups that can access instances. Therefore they act as a firewall on EC2 instance, by regulating: + +- Port access +- Authorised IP ranges +- Inbound traffic +- Outbound traffic + + + +Groups: + +- Can be attached to multiple insrtances +- Locked down to region/VPC combination +- Live outside Ec2 instances - they are their own standalone thing +- timeouts usually mean security group issue +- inbound traffic is blocked by default +- outbound traffic is authorised by default +- you can attach security groups to more than one instance + +## Ports + +These are the ports you must know: + + + +## SSH + +SSH is a CLI that can be used on Mac and Linux and Windows V > 10 (or PuTTy) below V10 + +EC2 instance connect also allows connection to your EC2 instances. + +AWS gives you the user EC2-user already, so the SSH command to login to the server has the following components: + +1. ssh ec2-user@<YOUR-PUIBLIC-IP-ADDRESS> +2. you need to use the .pem file (which contains a private key) using the -i flag (identity file flag) + +The full command: `ssh -i EC2Tutorial.pem ec2-user@44.201.88.145` + +> With all EC2 instances, if you experience a timeout, either when using SSH or otherwise, it's usually a security group issue + +## EC2 instance connect + +You can do all this in the browser, without managing keys by going to SSH Instance Connect. + +## IAM Roles for EC2 instances + +You should always manage EC2 instance access through IAM roles, not by adding your credentials directly into the instance using `aws configure` as this data can be accessed by other users on the instance. So instead, attach an IAM Role to the EC2 instance and manage service access through role policies. + +## EC2 pricing + +There is different EC2 pricing, which you can see below depending on what's needed: + +[ex2pricing](assets/sni.png) + +## IPV4 vs 6 + +AWS will charge for IPV6 ip addresses that go over 750 hours a month. So if you have more than 1 ip address it's likely you will incur costs. + +## EC2 controls + +There are a few ways to control the instances: + +- **Stop**: data is kept intact for next start. OS has to boot and can take time +- **Terminate**: data and setup is lost +- **Hibernate**: RAM is preserved, OS is not stopped / restarted. RAM state is written to file in volume. + +> Hibernation helps for saving RAM state, boot up fast and want long running processes. Can be no longer than 60 days. diff --git a/src/certifications/aws-solutions-architect/elasticache.md b/src/certifications/aws-solutions-architect/elasticache.md new file mode 100644 index 0000000..a17626b --- /dev/null +++ b/src/certifications/aws-solutions-architect/elasticache.md @@ -0,0 +1,53 @@ +# ElastiCache + +## Overview + +ElastiCache is Amazon's offering for managed Redis or Memcached. + +Caches are in-memory dbs with high perf and low latency. They allow quick access to data without having to reach the DB disk. + +This helps: + +- Latency +- Reduce load on DB resources +- Makes application stateless + +> AWS will take care of maintenance, patching, optimisations, monitoring, failure etc for you + +## Architecture + +Apps query the cache, but if it misses, then it will read from DB. + +Caches should have an invalidation strategy + + + +## Use session store + +Another good use case is for a session store for a user's session: + + + +## Elasticcache security + +- ElastiCache support IAM Auth for Redis +- IAM is only used for API level access, though +- There is also Redis auth, which allows you to create a user/pw combination for Redis cluster and is an extra layer of security +- Memcached supports SASL based auth + +## ElastiCache patterns + +1. **LazyLoading**: this involves writing all data to cache for quick access. Data can quickly become stale. +1. **Write Throgh**: When data is written to the the db, also write to cache +1. **Session store**: Store temporary data in cache (such as session info) + +## Redis use case + +> Important for the exam + +- Gaming leaderboards are computationally complex +- **Redis sorted sets** guarantees uniqueness and element ordering +- Each time an element is added, it's ranked in realtime, then added in correct order: + + + diff --git a/src/certifications/aws-solutions-architect/eni.md b/src/certifications/aws-solutions-architect/eni.md new file mode 100644 index 0000000..82aa12e --- /dev/null +++ b/src/certifications/aws-solutions-architect/eni.md @@ -0,0 +1,22 @@ +# Elastic Netowrk Interfaces (ENI) + +https://aws.amazon.com/blogs/aws/new-elastic-network-interfaces-in-the-virtual-private-cloud/ + +ENIs are a virtual network card that give EC2 instances network access. + +Each ENI can have: + +1. a private iPV4 (or more) +2. one elastic IPv4 +3. one public IPv4 +4. security groups +5. a MAC address + +They are AZ bound and can be moved to other EC2 instances. + +> Network cards are a hardware component that allow computers to connect to networks +> ENIs are outside of the EC2 instance, although they affect them they just aren't shared by the instance + +Each EC2 instance has one network interface. + +Moving an ENI between instances allows for quick failover. diff --git a/src/certifications/aws-solutions-architect/iam.md b/src/certifications/aws-solutions-architect/iam.md new file mode 100644 index 0000000..e8b821e --- /dev/null +++ b/src/certifications/aws-solutions-architect/iam.md @@ -0,0 +1,108 @@ +## Identity Access Management (IAM) + +This is a global service. + +A root account is created first, but it shouldn't be used thereafter, we should create new users instead. + +Users can be grouped. + +Groups only contain users, not other groups. + +A user can be in multiple groups, and users don't *need* to be in groups. + +## IAM permissions + +User permissions are defined as *policies*, which are JSON documents that manage user permissions. + +AWS follows the **least privilege principle**, meaning you don't give more permissions than a user needs. + +## Policy inheritence + +User inherit policies from the team(s) they belong to. But you can also create an **inline policy**, which is a policy just for one user, assuming they don't belong to a group. + +If a user belong to 2 groups, they will inherit the permissions from both groups. + +## IAM Policy + +Consist of: + +1. Version +2. ID +3. Statement + +Statements consist of ids, effects, principals, actions and resources. + +## Protecting your account + +There are two main ways to protect an account, a password policy and Multi Factor Authentication (MFA) + +### Password policy + +This policy defines what type of password users can store (character and length etc) but also how often the password resets, whether the user can use the same password more than once etc. + +### MFA + +This is an additional step during login and requires a user to type in an additional code once they have submitted their password. There are four ways to do this: + +1. A virtual device like Google Authenticator +2. A security key (U2F) device +3. Hardware key fob +4. Hardware key fob ( + +## Protecting your account + +There are two main ways to protect an account, a password policy and Multi Factor Authentication (MFA) + +### Password policy + +This policy defines what type of password users can store (character and length etc) but also how often the password resets, whether the user can use the same password more than once etc. + +### MFA + +This is an additional step during login and requires a user to type in an additional code once they have submitted their password. There are four ways to do this: + +1. A virtual device like Google Authenticator +2. A security key (U2F) device +3. Hardware key fob +4. Hardware key fob (for AWS GovCloud US) + +## IAM roles for services + +Sometimes we need services to perform actions for us. For example we can have an EC2 instance and we need that instance to manage our AWS account. + +To do this we use IAM Roles. + +For example, we create a role that is trusted by a service, such as EC2. This role may have permissions like accessing `IAMReadOnlyAccess`. Now our EC2 instance can read our IAM. + +## Security tools + +### IAM Credentials Report + +This lists all users and their credentials. + +### Access Advisor + +This lists individual users and the permissions granted to them, including them they last accessed the services. This allows you to understand which services the users has access to and which you might want to revoke. + +## Best practices for IAM + +- Don't use root account except for setup +- 1 physical users should = one AWS user +- Assign users to groups and assign permissions to groups +- Create strong pw policy +- Use and enforce MFA +- Create Roles for giving permissions to AWS services +- Use access keys for programmatuc access +- Audit permissions using Access Advisor +- Never share IAM user & access keys + +## Overview + +- Users are physical users and each have their own access +- Groups contain users only +- Policies are JSON docs with permissions +- Roles give features AWS access +- Security: MFA + p/w policy +- CLI/SDK are programmatic acesss +- Access keys : give programmatic access +- Audit users with credential reports and access advisor diff --git a/src/certifications/aws-solutions-architect/infra.md b/src/certifications/aws-solutions-architect/infra.md new file mode 100644 index 0000000..85268d6 --- /dev/null +++ b/src/certifications/aws-solutions-architect/infra.md @@ -0,0 +1,21 @@ +You can check out the AWS infrastructurnd regions here: [https://aws.amazon.com/about-aws/global-infrastructure/regions_az/](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/) + +It's a global infrastructure, meaning we can deploy apps to be global + +## Regions + +Regions have names, e.g us-east-1 + +A region is a cluster of data centres + +Most services are region scoped. + +Some factors in choosing regions could be: + +- Compliance +- Latency +- Service availability (some regions don't have certain services) + +Each region has 'zones', which are separate data centres from each other and are isolated from disasters. + +Not all services are available in all regions, see here: [https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/?p=ngi&loc=4](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/?p=ngi&loc=4) diff --git a/src/certifications/aws-solutions-architect/intro.md b/src/certifications/aws-solutions-architect/intro.md new file mode 100644 index 0000000..fc39dfd --- /dev/null +++ b/src/certifications/aws-solutions-architect/intro.md @@ -0,0 +1,23 @@ +Amazon Web Servbices (AWS) is a cloud provider. + +It provides servers and services on demand that can easily scale. + +It's in direct contrast to on-premise solutions that require ordering servers and setting them up manually. + +It powers a lot of well known websites, like Netflix.com, which rents all their servers from AWS. + +## Facts + +- AWS has 90b in revenue +- 31% of cloud market +- 1m active users + +## Use cases + +Almost any use case you could want. + +- Hosting websites +- Backups and storage +- Big data analytics etc etc + + diff --git a/src/certifications/aws-solutions-architect/ips.md b/src/certifications/aws-solutions-architect/ips.md new file mode 100644 index 0000000..8504690 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ips.md @@ -0,0 +1,23 @@ +# IPs + +## IPV4 vs IPV6 + +IPV4 are "classic" IP addresses, but they are running out, so the internet is moving to IPV6. + +The IPV4 address we get with our EC2 instance will be enough. + +The format is as follows: [0-255].[0-255].[0-255].[0-255] allowing for 3.6b IPV4 addresses + +## Private VS public IPs + +Public IPs allow servers to be accessed via the internet. Whereas private IPs can only be accessed internally from the same network. Private networks can interact with the WWW using an internet gateway (proxy) + +Private IPs can repeat, whereas public IPs cannot. + +## Elastic IPs + +AWS allows Elastic IPs, which are static IP addresses you can keep and then port between services. So the IP remains static and can be applied to different services. AWS gives you 5 of these. Each can only be attached to an instance one at a time. + +> Avoid elastic IPs, instead register a DNS name to a random public IP. Or use a load balancer. + + diff --git a/src/certifications/aws-solutions-architect/load-balancing-ELB.md b/src/certifications/aws-solutions-architect/load-balancing-ELB.md new file mode 100644 index 0000000..2dedf72 --- /dev/null +++ b/src/certifications/aws-solutions-architect/load-balancing-ELB.md @@ -0,0 +1,188 @@ +## Load balancing with Elastic Load Balancer (ELB) + +Load balancing refers to distributing user traffic to multiple instances. + +A load balancer sits in front of your instances and distibutes traffic depending on instance load. + +This diagram explains it well: + + + +## Why? + +- Spreads load +- Single point of access using DNS +- Failure handling +- Load balancers can check health of instances + - Load balancers check if downstream server will reply to requests +- HTTPS support +- High availability +- Can separate public and private traffic + +## Elastic Load Balancer (ELB) + +Managed service that AWS maintains. No need to configure it yourself. Integrates with: + +- EC2, EC2 auto scaling groups, ECS +- AWS Cert Manager, CloudWatch +- Route 53, AWS WAF, AWS Global Accelerator + +### ELB types + +Different generation of EBS: + +1. Classic Load Balancer (old gen) +2. Application Load Balancer (v2) +3. Network Load Balancer (v2 new gen) +4. Gateway Load Balancer - 2020 and newest + +Load balancers will allow access from anywhere, but the connection betweem EBS and EC2 instances is managed through security groups so EC2 access is restricted. + +## Load balancer types + +> It's important to note that ELB is the overall service, but there are different types of load balancers for different purposes, as noted below + +### Application Load Balancer (ALB) + +This load balancer works at the HTTP level and allows for: + +- HTTP load balancing +- Container load balancing +- HTTP/2 and WebSocket +- Redirects (HTTP > HTTPS) + +Load balancing can be based on: + +- Path based. /users or /posts for e.g +- Hostname (subdomains, for e.g) +- Query strings and headers + +> ALB is good for microservices where you want to balance load between services and each service might hit a different endpoint, subdomain, container etc + + + +## Target groups + +Path routing, for example, determines which target group (a set of resources) to route request to. Target groups can be: + +- EC2 instances +- ECS tasks +- Lambda functions +- IP addresses + +For example query param target group routing: + + + +# Good to know: + +- ALBs get a fixed hostname +- App servers don't see client IP directly, instead it's passed through headers + +### Network Load Balancer (NLB) + +This is a lower level load balancer (Layer 4) - affecting the TCP and UDP layer. + +It has: + +- Ultra low latency +- Handles millions of RPS +- 1 static IP per AZ and supports supports Elastic ips + +### Gateway Load Balancer (GLB) + +- A way to scale and manage a fleet of 3rd party network applications in access +- Example: firewalls, intrustion detection +- It is a way to route all traffic through a gateway +- You can think of it as an additional layer between traffic and your applications. It allows you to analyse network traffic and possibly drop +- Operates at Layer 3 (network layer) +- It has the following functions: + - Transparent Network Gatewau - single entry/exit for all traffic + - Load Balancer - distributes traffic to virtual applications +- It uses GENEVE protocol on port 6081 + +> GLB allows you to analyse all traffic and drop it if necessary. + + + +Target groups can be: + +- EC2 instances +- IP addresses + +## Sticky Sessions + +Sticky sessions is a way to keep the same user on the same instance even when using a load balancer. + +Usually a load balancer will distribute traffic among different instances, but if you want to keep the same user on the same instance, for example to keep their session information, you can use sticky sessions. + +It's controlled using a cookie, which is either: + +- A custom cookie +- A default cookie set by Sticky Sessions + +## Cross-Zone Load balancing + +Cross-zone load balancing allows you to distribute load balancer traffic across different AZs. It can be turned on and off per load balancer. + + + +If it's turned on, all traffic will be spread evenly across AZs. Otherwise it won't. + +It's automatically on for ALBs, but needs to be turned on manually for NLBs and GLBs and will incur a charge for the latter two. + +## Connection Draining + +This gives the request time to complete while instance is de-registering or "unhealthy". + +It will stop sending requests to instances that are de-registering. + +Can be set from 1 to 3600 seconds. + +## Auto Scaling Group (ASG) + +An ASG adds or remove more instances depending on user load. It's referred to as: + +- Scale out (adding EC2 instances) +- Scale in (removing EC2 instances) + +They are free! You only pay for the infra you get. They also work OOTB with load balancers, which connect automatically to new instances added. + +You get to define: + +- **Minimum capacity** +- **Desired capacity** +- **Maximum capacity** + +You define a *launch template* that contains all EC2 info that are used as parameters to launch new instances. + +### CloudWatch alarms + +When alarms are triggered (avg CPU or custom metric) the Auto Scaling Group can be enacted. + +### Auto Scaling Groups - Scaling Policies + +Scaling policies allow us to define under what conditions to scale. + +#### Dynamic Scaling + +- *Target Tracking Scaling*: define metric (for e.g CPU usage) and ASG will scale out or in to match that metric +- *Step Scaling*: This scales depdning on CloudWatch alarm trigger, for e.g CPU > 70%, add 2 units +- *Scheduled scaling*: Scale based on times. Based on usage patterns + +#### Predictive Scaling + +ASG analyses historical load, forecasts load and then scales + + +## Good metrics to scale on + +- CPU utilisation +- RequestCountPerTarget: make sure requests per instance is stable +- Average network in/out +- Any custom metric! + +## Scaling cooldown + +After removing/adding instances, which is a 5 minute cooldown where no instance changes can be made. Allows for instance stablisation + diff --git a/src/certifications/aws-solutions-architect/route-53.md b/src/certifications/aws-solutions-architect/route-53.md new file mode 100644 index 0000000..8c9371d --- /dev/null +++ b/src/certifications/aws-solutions-architect/route-53.md @@ -0,0 +1,167 @@ +# Route 53 + +## What is DNS? + +Domain Name Servers (DNS) are a way to resolve website URLs to IP addresses. + +For example translating www.google.com to it's IP address of 172.217.18.36. + +## URL breakdown + +This is the breakdown of a URL: + + + +## How DNS works + +The process of resolving an IP address takes many steps, including reaching out to many servers: + + + + +## Amazon Route 53 + +This is Amazon's DNS service and domain registrar. + +Requests to websites registered with Route 53 will go to Amazon's Route 53 DNS. + +## Route 53 DNS records + +DNS records are instructions stored on DNS records that allow us to connect domain names to IP addresses. Route 53 is *authoritative*, meaning you have control over your DNS records. + +Route 53 supports the following records: + +- **Must know**: A / AAAA / CNAME / NS +- **Advanced**: CAA / DS / MX / NAPTR / PTR / SOA / TXT / SPF / SRV + +### Record types + +- **A**: maps a domain name to an IPv4 address +- **AAAA**: maps a domain name to an IPv6 address +- **CNAME**: maps a hostname to another hostname. C stands for "canonical" and CNAMEs allow you to create aliases for your domain names. For example, you could point example.domain.com to domain.com to resolve the IP address, instead of to the IP address itself. The IP resolution will happen at domain.com +- **NS**: Name Servers for the hosted zone. This controls how traffic is routed for the domain. + +## Route 53 - Hosted Zones + +Hosted Zones are containers that define how to route traffic to a domain and subdomains. For example you can have: + +- **Public Hosted Zones**: records that define how to route public domain names +- **Private Hosted Zones**: records that define how to route private domain names + + + +## TTL (Time to live) + +TTL is basically how long the client should cache DNS records. You can set TTL for each DNS record, which tells the browser to not make another DNS request once it has the record for that duration. + + + +## CNAME Vs Alias + +### CNAME + +CNAME is a way to point one domain to another domain `app.mydomain.com => blabla.amazon.aws.com`, for example. + +However, this only work for non root domains. `mydomain.com` wouldn't work, for example. + +### Alias + +This points a hostname to an AWS resource `mydomain.com` to `myalb.amazonaws.com`. + +Here we are pointing a root domain to an AWS resource. + +This works for both root and non root domains. + +## Routing policy + +The routing policy is the policy that defines how DNS responds to queries, so how the IP name is resolved for a domain name. + +### Simple routing + +This just routed traffic to a single resouce. For example `foo.example.com` is routed to `11.22.33.44`. + +If there are multiple IP addresses for the same records, the client will randomly choose one. + +- If alias is enabled, you can only specify one resource +- No health checks possible + +## Weighted routing + +Weighted routing allows you to define a relative amount of traffic to be routed to a resource. + +AWS will apply a percentage of traffic to that resource correlating to the weight set. + +For example: + +- weighted.example.com (some ip) -> 10 (some ip) +- weighted.example.com (some ip) -> 70 (some ip) +- weighted.example.com (some ip) -> 20 (some ip) + +70% of traffic will be sent to the second resource. + +A weight of 0 means don't sent traffic. + +> Use cases can be load balancing, testing new apps on a % of traffic etc + +### Latency-based + +Routes requests from the user to the nearest resource + +> This is useful when latency is a priority + +## Health checks + +Health checks check the status of our instances and whether they are running or down. + +They are for **public resources** only. The health checkers come from outside our instances, so our resources need to be public facing to be evaluated by the health checkers. + +There are three types: + +1. Health checks that monitor an endpoint (app, server etc) +2. Health checks that monitor other health checks (Calcualted Health Checks) +3. Health checks that monitor CloudWatch Alarms (e.g DynamoDB, alarms on RDS) + +### Endpoint monitoring + +- 15 global health checkers will monitor endpoint health. +- If endpoint responds with `2**` or `3**`, then it's considered **healthy**, otherwise it's **unhealthy**: + + + +### Calculated Health Checks + +- Combine multiple health checks into one +- Uses operators like **AND**, **OR**, **NOT** + + + + +### Private hosted zones + +You can create a CloudWatch Metric and associated CloudWatch Alarm then create a Health Check that checks the alarm itself + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/certifications/aws-solutions-architect/scalability-availability.md b/src/certifications/aws-solutions-architect/scalability-availability.md new file mode 100644 index 0000000..4ed4a10 --- /dev/null +++ b/src/certifications/aws-solutions-architect/scalability-availability.md @@ -0,0 +1,31 @@ +## Vertical vs horizontal scalability + +### Vertical + +This means adding more power to existing instance. For example turning EC2 instance from a T2 to a T3 instance - so adding more compute. + +- There are hardware limits to how much this can scale +- Common use case fo non-distributed systems like a database +- RDS and ElastiCache can scale vertically + +> AWS terms: scale up / down + +### Horizontal + +Involves adding more instances for the application. Instead of increasing capacity of existing machines, we add new machines. + +So going from one machine to two machines for the same application is an example of horizontal scaling. + +- Common use case for web apps +- Implies distributed systems + +> AWS terms: scale out / in + +## Availability + +This refers to application uptime. It usually involves running same application across multiple AZs so that if one data centre crashes, your app will be active in another so there is no downtime. + +AWS examples: + +- Auto Scaling Group multi AZ +- Load Balancer multi AZ diff --git a/src/certifications/aws-solutions-architect/ssl-tls.md b/src/certifications/aws-solutions-architect/ssl-tls.md new file mode 100644 index 0000000..49e6956 --- /dev/null +++ b/src/certifications/aws-solutions-architect/ssl-tls.md @@ -0,0 +1,34 @@ +## SSL / TLS Basics + +SSL refers to Secure Socket Layer and is a way to encrypt network traffic that is in flight. + +For AWS it refers to encrypting traffic between the client and the load balancer. Traffic between the load balancer and instances are handled over plain HTTP. + +The S in HTTPS refers to 'secure', which means the traffic is using SSL. + +Nowadays, traffic is encrypted using TLS (Transport Layer Security), which is a newer version of SSL, but people still refer to it as SSL. + +SSl certs are issued by a Certificate Authority (CA) and have an expiration date. + + + +## Load Balancer and SSL + +- LB uses an X.509 Certificate +- ACM (AWS Certificate Manager) manages certs +- You must set a default SSL cert +- Clients can use SNI (Server Name Indication) to specify the hostname they reach + + +## Server Name Indication (SNI) + +- This helps manage multiple SSLs on one Server +- Client must indicate server hostname to get the right SSL + + + +## Load Balancer Support + +- *CLB*: supports 1 SSL +- *ALB*: supports multiple with SNI +- *NLB*: support multiple with SNI |
