HAMISPA - MISP Threat Intelligence Summit 0x05

Deploying a high resiliency, high availability MISP cluster in AWS.

hack.lu 2019, Luxembourg

Transcript

1. MISP Threat Intelligence Summit 0x05 | Tiago Faria <0xtf@3coresec.com> HAMISPA High Availability MISP in AWS
2. $ whoami - Tiago Faria - Work @ 3CORESec - Maintainer of MISP-Cloud github.com/MISP/misp-cloud
3. $ cat about.txt - Not really about AWS - Present scenarios for HA of MISP - Experiments that worked and failed - Notes for deployment
4. $ cat not-about.txt - Step-by-step guide for deployment - Step-by-step guide to install MISP - A course on Amazon Web Services - Evidence that this is the best deployment that can be done
5. $ eog why.png
6. $ cat why.txt Hopefully useful for anyone looking to increase resiliency and availability of their MISP deployments. A public reference to high availability in MISP deployments.
7. $ cat about-hamispa.txt | more A MISP cluster in a high availability, resilient environment, that leverages AWS services for performance, security and cost.
8. --MORE-- - HA & DR for Database - Cluster for MISP codebase - Shared file-system between cluster nodes - Load-balancing between cluster nodes - Auto-scaling for availability and performance - Use encryption for both in-transit and at-rest - Leverage WAF and DoS protection
9. $ eog hamispa-architecture.png
10. AWS 101
11. $ eog hamispa-architecture.png
12. - AWS Virtual Private Cloud (AWS VPC) - VPC with 2 private subnets and 2 public subnets (different availability zones) - DHCP options configured accordingly - IGW for public subnet - NAT gateway from private subnets $ cat aws-vpc.txt
13. $ eog hamispa-architecture.png
14. - AWS Relational Database Service (AWS RDS) - Configure, manage and scale databases - Automation for hardware provisioning, setup, patching and backups - Supports several engines (including MySQL) - DR via auto-updating DNS endpoint - Read replica (more on this later) $ cat aws-rds.txt
15. $ eog hamispa-architecture.png
16. - AWS Elastic File System (AWS EFS) - Fully-managed NFS file system - Amazon maintains mount helper for most distros - Easy to mount via TLS - No upfront requirements for provisioning - Will hold MISP codebase (/var/www/MISP) $ cat aws-efs.txt
17. $ eog hamispa-architecture.png
18. - AWS Elastic Compute Cloud (AWS EC2) - Compute capacity (think VMs) in AWS - Friendly GNU/Linux ecosystem - These instances will be our cluster nodes - Part of our Auto-Scaling Group (ASG) - Holds AWS SES configuration, PHP, Apache and MISP in AWS EFS mount point (/var/www/MISP) - Will be the source for the AMI that will be used on auto-scaling by other cluster nodes $ cat aws-ec2.txt
19. $ eog hamispa-architecture.png
20. - AWS Elastic Load Balancing (AWS ELB) - Entry point for access to our EC2 instances - Distributes traffic across multiple targets - Keeps an eye on instance health - Interacts with our auto-scaling group - Source for our CDN? $ cat aws-elb.txt
21. $ eog hamispa-architecture.png
22. - AWS Elastic Compute Cloud (AWS EC2) - Bastion-host lives in the public subnet - SSH key and Security Group for single IP - MFA for login - Turned on/off as needed $ cat aws-bastion.txt
23. $ eog hamispa-architecture.png
24. - AWS Web Application Firewall (AWS WAF) - Support for SQL Injection; XSS; Custom rules - Integrated with both ELB and Cloudfront (CDN) - One-click deployment $ cat aws-waf.txt
25. $ eog hamispa-architecture.png
26. - AWS Shield - Managed Distributed Denial of Service protection - Inline/Always-on detection and mitigation - Turned on by default (AWS Shield Standard) - AWS Shield Advanced supported for both ALB and Cloudfront $ cat aws-shield.txt
27. $ eog hamispa-architecture.png
28. - AWS Route53 - HA Managed DNS Provider - Several routing policies: Geo DNS, Geo-Proximity, Round Robin - Health Check: DNS Failover $ cat aws-r53.txt
29. $ eog hamispa-architecture.png
30. - AWS Key Management System (AWS KMS) - Create and manage cryptographic keys - Manage encryption in several AWS services - HSM for FIPS 140-2 validation $ cat aws-kms.txt
31. Putting it all together
32. $ eog hamispa-architecture.png
33. - Create VPC & prepare the firewall (SGs) - Create EFS share - Create instance in RDS - Create OS and mount EFS; Configure; Tweak it - Create an AMI that will be used for future nodes - Install MISP in EFS - DB Connect to RDS - Create ASG $ cat INSTALL.txt
34. - Configure ALB - Configure Health Checks and Conditions - Configure DNS - Issue certificate based on DNS - Enabled WAF for ALB - Publish it to AWS Cloudfront (CDN)? - Enjoy! --MORE--
35. Chaos in HAMISPA
36. - Health check for Apache failure - Recommended: use simple .html file (ping.html) - Health check type: Elastic Load Balancer - ELB Health Check fails: - New instance is spawned based on AMI - Failed instance is terminated $ cat chaos-apache.txt
37. Slide demo video for export to PDF: https://www.youtube.com/watch?v=Ni8IygluM9g
38. - CPU Percentage failure - Health check type: EC2 Instance - EC2/Cloudwatch Failure for stressed CPU: - New instance is spawned based on AMI - Failed instance is terminated $ cat chaos-cpu.txt
39. Slide demo video for export to PDF: https://www.youtube.com/watch?v=Ohuko2_4H0E
40. - Instance termination failure - Health check type: Elastic Load Balancer - EC2/Cloudwatch Failure required instances: - New instance is spawned based on AMI $ cat chaos-termination.txt
41. Slide demo video for export to PDF: https://www.youtube.com/watch?v=ZKYLYYNejGM
42. Future Improvements
43. - More effective proxying (move to NGINX?) - New ASG for AWS RDS Read-replica for programmatic access - AWS Elasticache (Redis) - DNS Failover for different ALB $ cat hamispa-ideas.txt
44. Technical Details github.com/0xtf/HAMISPA
45. Questions? @0xTF $ cat questions.txt Thank you!