Dmytro Konstantynov

🚨 It's not a bug; it's by design. A brief summary — the article's author, after experimenting, received a bill of over $1,000 for an empty (!) private (!) S3 bucket. After reading the documentation, he discovered that indeed, this could happen. Moreover, technical support confirmed it - yes, it's intended behavior. Bucket owner is charged for the request under the following conditions: • Request authentication fails (HTTP code 403). • The request is anonymous (HTTP code 403). How then can one protect themselves? • Generate long S3 bucket names from random characters (free). • Use AWS Shield Advanced ($3k/month). ✘ Write anything in S3 bucket policy — won't help (see Request authentication fails). ✘ Allow access only from your own VPC (see the previous point). • Add to the Readme "I'll find you by IP!!" (affordable). But why?!? S3 is a very old service, some even think it was the first (actually, the first was SQS). When it was designed, privacy wasn't an issue, the reverse was the problem — making it public. By design, the S3 service and, importantly, the S3 API are public. The concept of VPC, and later the concept of "private buckets," appeared significantly later, in 2011. So, it's important to note that this is more of a "marketing" name because by design, the buckets are public or can become so, and are also unique. In short, it's impossible to completely and freely protect yourself from Denial-of-Wallet attacks by definition. And is it really that bad? No. The problem has always been there. AWS has ways to detect and resolve it, including with the help of technical support. Accidentally generating significant billing isn't easy, as it would have to be not millions, but billions of requests. Plus, of course, you must have a budget alert set up. 😀 Overall, AWS has room for improvement. And public discussion of an old architectural problem is an excellent incentive. #aws #awscommunity #awscommunitybuilders #devops https://lnkd.in/gNyuUpgx

51

10 comentários

Terraform's Import block functionality will help give a lot of companies the ability to bring their existing infrastructure under a unified code base. Now with the ability to use for_each inside their import block this video shows a cool way to utilize that feature to build on top of your existing pipelines without having to re-engineer anything.

4

DevoxxGenie: Your AI Assistant for IntelliJ IDEA https://lnkd.in/dJVrEnjx DevoxxGenie is a fully Java-based LLM code assistant plugin for IntelliJ IDEA, designed to integrate with local LLM providers and cloud-based LLMs. In this blog, you will learn how to get started with the plugin and get the most out of it. Enjoy! Introduction AI coding assistants are getting more and more attention. Multiple cloud services are available and IDEs provide their own services. However, with this approach, you are tied to a specific vendor and most of the time, these services are cloud based only. What should you do when your company policy does not allow you to use these cloud-based services? Or what do you do when a new, more accurate model is released? When using IntelliJ IDEA, you can make use of the DevoxxGenie plugin, created by Stephan Janssen, founder of the Devoxx conference. The plugin allows you to choose between several cloud based and local LLMs. You are not tied to a specific cloud vendor or a specific model. This is important because every few weeks a new model is released, which outperforms previous ones.

2

Last weekend, in addition to my blog and courses, I started to write a book about Cloud Engineering focused on #GCP. So far I’ve planned to include CLI, Docker, Terraform, Pipelines and the most used GCP resources. What would you like me to include most in detail? 🤓📖

9

5 comentários

I wrote a new blog post about my experience at ApacheCon - ASF Events Community Over Code, Outreachy and what it's like working at Percona, and the cool cloud-native tool for databases on Kubernetes that we are working on right now: Percona Everest, open-source! 🤗 👩‍💻 💃 🕺 https://lnkd.in/djdCxXea If you have any questions about any of these topics, send me a direct message so we can have a virtual coffee! ☕ #databases #cloudnative #opensource #kubernetes

28

1 comentário

I just published a new article about "How to Connect a Private Cluster Using a Jumpbox in Terraform" in medium. –You can check it out here: https://lnkd.in/d3R3kSc4 #Terraform #GCP #Kubernetes

17

⏳ Just 2 more weeks to register for my next online workshop about Fetching Best Practices with Spring Data JPA. 🌼 If you want to speed up your Spring data access layer, then you should definitely join. https://lnkd.in/dcYZn4fJ

33

Adolfo Ochagavía approach of using S3 as a container registry is groundbreaking. It's incredibly fast, outperforming ECR by 5-8 times. Definitely worth exploring. https://lnkd.in/dF7EGVbz #s3 #aws #awsclod #ecr #docker #eks #ecs #k8s #storage

11

Grafana is extremely creative in its approaches, but it is not always easy to understand the meaning of innovations 💭 ... there some useful information for the mimir project 🥕 on how to migrate from consul to quorum memberlist and why memberlist labels matter 💎 #monitoring #infrastructure #devops #grafana #observability

2

⚙ Using S3 as a container registry is 5-8 times faster than ECR! Very interesting approach by Adolfo Ochagavía. Definitely worth a look. #aws #awscommunity #awscommunitybuilders #devops https://lnkd.in/dHAQz2EN

1.388

104 comentários

Kubectl Port-forward Flow Explained From the content: Recently, I joined a discussion about how the kubectl port-forward command works, which caught my attention because I have an app that improves some aspects of the native kubectl port-forward. I made a public Mermaid chart to show the complete process of executing this command. ...[Read more]

12

#RabbitMQ Streams is a feature introduced to provide high throughput, low latency, and high availability for messaging and event streaming. It was officially introduced in RabbitMQ version 3.9, released in July 2021. RabbitMQ Streams is designed to handle large volumes of data efficiently, providing stream semantics similar to those found in other stream processing platforms like Apache Kafka. Key Features: - High Throughput and Low Latency. - Stream Semantics: Unlike traditional RabbitMQ queues, which store messages and are consumed and removed, RabbitMQ Streams retain messages for a specified period or until they reach a specified size. Consumers can read messages at any point within the stream, allowing for replayability and time-based processing. - Efficient Storage: Streams use efficient disk storage techniques to handle large volumes of data. They store messages in segments and use write-ahead logging to ensure data durability and consistency. - Consumer Offsets: Streams track consumer offsets, enabling consumers to resume processing from where they left off. This is similar to how Kafka handles consumer offsets, allowing for robust and fault-tolerant consumption. - Advanced Routing and Filtering: Streams support advanced routing and filtering capabilities, allowing for selective consumption of messages based on specific criteria. - Horizontal Scalability: Multiple brokers can be used to distribute the load, and stream segments can be replicated for high availability. - Backwards Compatibility: Designed to be compatible with existing RabbitMQ infrastructure, allowing users to integrate streams into their current RabbitMQ deployments. Use Cases: - Event Sourcing: Streams are well-suited for event sourcing architectures, where the complete history of changes to an application's state needs to be stored and replayed. - Real-time Analytics: Applications that require real-time data analytics can benefit from the high throughput and low latency characteristics of RabbitMQ Streams. - Log Aggregation: Collecting and processing logs from various sources in a centralized manner can be efficiently handled using RabbitMQ Streams. - Data Pipelines: Building data pipelines that require reliable, scalable, and fault-tolerant message delivery. 📣 WhatsApp Channel: https://lnkd.in/gxQfpuGD

3

For the Sake of Search episode two is out! This week Nicholas Khami and I talk about his experience building Trieve and rebuilding the Hacker News search. And of course we also talk through the #Elasticsearch license change! How could we not right? Make sure to catch it here: https://lnkd.in/eUSAS7hR Or on any of your favorite podcasting providers!

16

2 comentários

🔍 Unlocking Prometheus Power: Your Go-To Guide for Setting Up Prometheus and Prometheus Node Exporter. Ready to Elevate Your Monitoring Game? Let's Dive In! 🚀 #DevOps #Prometheus #NodeExporter #MonitoringMagic

4

I get asked every week: how do we know if something is wrong? The truth is: most of us wait until it's too late. Kubernetes is no magic box. Things will fail. To be even more precise: Kubernetes was built to fail. Failing nodes. Crashing pods. If done correctly, Kubernetes will handle it all for you. But without the proper preparation, you are nowhere. Because Kubernetes does not know what is important to you. You need to tell it. Much of that is configuration (replicas, PDBs, TopologySpreadConstraints, etc.). But the other part is observability. Alerts. Have Kubernetes tell you immediately if something you care about is not OK. So you can act. Before it's too late. Not when your customer calls you. The question then becomes: what do YOU care about? CPU high for a few minutes? Probably not. Autoscaler shutting down a node? Let me sleep. But a certificate not being renewed? Why the hell not?! ExternalSecret not being synced? Tell me why. OS patches not being installed due to failing reboots, that's not good. [Fill in here what makes you worry at night] So do not only design for failure, also design your observability stack for failure. My life-saving alerts: CertManagerCertExpirySoon - avg by (exported_namespace, namespace, name) (certmanager_certificate_expiration_timestamp_seconds - time()) < (21 * 24 * 3600) ExternalSecretSyncError - externalsecret_status_condition{condition="Ready",status="False"} == 1 ThanosCompactHalted - thanos_compact_halted{job=~".*thanos-compactor.*"} == 1 ArgoAppNotSynced - sum by (name) (count_over_time(argocd_app_info{autosync_enabled="true",container!="pushgateway",name!="appgroup-apps",name!="appgroup-platform",name!="bootstrap",sync_status!="Synced"}[1m])) > 0 NGINXTooMany500s - (sum by (ingress) (rate(nginx_ingress_controller_requests{status=~"5.+"}[2m]))) / (sum by (ingress) (rate(nginx_ingress_controller_requests[2m]))) > 0.05 RebootRequired - max(kured_reboot_required) != 0 PodOOMKilled - kube_pod_container_status_last_terminated_reason{reason="OOMKilled"} unless kube_pod_container_status_last_terminated_reason{reason="OOMKilled"} offset 1m Bonus 1: use that freaking Watchdog! It's not there for nothing. Send it to an external health check system and get informed when your monitoring is down. Bonus 2: at Pionative we have an alert that indicates when we have too many on-demand nodes as opposed to spot instances. Helps to keep costs down. — Like this thread? Please let me know which alert saved your life in the comments below. Also, follow me for more insights.

41

2 comentários

Read “Predicting Car Prices with #FastAPI, #Streamlit, #MLflow, #Kafka, and #Debezium: A Practical…“ by Stefentaime on Medium: https://lnkd.in/dTwgcyCQ

2

📢 On the 27th of November, I'm going to run a 3-hour workshop about Fetching best practices when using Spring Data JPA. 💡 If you want to learn how to speed up your data access layer, then you should definitely join. https://lnkd.in/dcYZn4fJ

25

My latest article about how to generate a simple end-to-end project with HuggingFace, Docker, and FastAI is now live on @kdnuggets Go check it out now 👇🏻 https://lnkd.in/dYWRHJA4

2

1 comentário

𝐒𝐭𝐨𝐩 𝐥𝐨𝐬𝐢𝐧𝐠 𝐭𝐢𝐦𝐞 (𝐚𝐧𝐝 𝐬𝐚𝐧𝐢𝐭𝐲) 𝐛𝐲 𝐟𝐨𝐫𝐠𝐞𝐭𝐭𝐢𝐧𝐠 𝐭𝐡𝐞 -𝐧 𝐟𝐥𝐚𝐠!! If you ever need to use namespaces in Kubernetes - please immediately install the kubens cli. I get annoyed with how people introduce bugs in the system just because they forgot to add -n flag. Checkout kubectx + kubens here: https://lnkd.in/eHPHpYYK #devops #kubernetes

25

1 comentário