Enterprise cloud migration sounds pretty straightforward until you actually need to do it. Your weekend plans change real quick when autoscaling configurations break, databases hit connection limits, and services try to write to the same queue at the same time. At Softeta, we’ve helped our business clients deal with these sort of situations. They taught us quite a bit about how to anticipate and prevent disasters.
We’re sharing this because most articles about scalable cloud infrastructure aren’t really helpful. They talk about architecture patterns in the abstract, draw clean diagrams and move on. In actuality, you’re often forced to make tradeoffs you’re not proud of. You inherit systems that were “temporary” four years ago. You tend to pick tools based on what your team already knows rather than what’s optimal on paper.
So this article is about the messy, real version of migration. What we’ve actually learned building and running cloud infrastructure for enterprise clients who process millions of transactions with SLAs breathing down their necks.
Why enterprise scaling is HARD
Scaling a startup’s infrastructure and scaling an enterprise application have almost nothing in common. I know people say this a lot, but I don’t think they explain why very well.
At a startup, if your service goes down for 20 minutes, you apologize on x.com and move on. At an enterprise, 20 minutes of downtime might violate a contractual SLA, trigger a financial penalty and require an incident report. That report gets reviewed by people who’ve never seen a terminal, by the way.
Then there’s the compliance requirements that dictate where your data can physically live. The legacy ERP system from 2009 that your entire order pipeline depends on. The fact that your “cloud migration” actually means running half your services on AWS and half on a VMware cluster in a colocation facility in Frankfurt (because legal said so).
Enterprise cloud scalability means handling traffic spikes while dealing with all of that.
Architecture decisions that we keep coming back to
The biggest recurring mistake we see is teams trying to scale a monolith by upgrading to bigger machines (vertical scaling). Everyone high fives when you go from an m5.xlarge to an m5.4xlarge. A few months later they’re back in the same spot but now the AWS bill is 4x what it was.
Some form of service decomposition is usually where you end up. I’m careful about calling it “microservices” because that word carries a lot of baggage now. What I mean is: separate the things that need to scale independently. Your payment processing service and your PDF report generator don’t need to be in the same deployable unit sharing the same database connection pool.
Event-driven architecture has been the biggest win for our clients when it comes to scaling enterprise systems. The idea is that instead of service A making a synchronous HTTP call to service B (and both of them failing together), service A drops a message on a queue and service B picks it up whenever it’s ready. Kafka and Amazon SQS are what we reach for most often. This gives you natural backpressure, meaning if service B is slow, messages just queue up instead of cascading failures everywhere.
Separating databases per service is the other big one. And yes, I know –everyone hates hearing this because it means you can’t just JOIN across your entire data model anymore. You have to deal with eventual consistency. Your reporting team will complain for sure. But the alternative is a single Postgres instance that becomes the bottleneck for dozens of different services, and that’s simply worse. We usually pair a relational database for transactional stuff with something like DynamoDB or Elasticsearch for the read heavy workloads.
An API gateway in front of everything rounds this out. One place to handle auth, rate limiting, routing. For enterprise apps that serve a mobile client, a web app, a partner API, and an internal admin tool, you need this. Kong and AWS API Gateway are what we’ve used most.
Your infrastructure isn’t in code? Then it’s not real
I have a hard rule about this. If you can’t tear down your entire production environment and rebuild it from a Git repo, your infrastructure isn’t production-ready. Full stop.
We use Terraform for almost everything. Some teams prefer Pulumi, which is fine. AWS CDK has its fans. The tool matters less than the practice: every single piece of infrastructure should be defined in version-controlled files. Every change should go through a pull request. Someone should review it before it gets applied.
The number of enterprises I’ve seen where one senior engineer runs terraform apply from their laptop, and they’re the only person who knows the current state of production… it’s scary. That person goes on vacation and suddenly nobody can make infrastructure changes for two weeks.
GitOps has been a good answer for the Kubernetes side of things. We use ArgoCD on most projects. The concept is that your Git repo is the single source of truth. If something is running in the cluster that doesn’t match what’s in Git, ArgoCD detects the drift and either alerts you or fixes it, depending on how you’ve configured it. It’s removed a lot of the “who changed what and when” mystery that used to plague our deployments.
Kubernetes: worth it, but expensive to operate
I have a complicated relationship with Kubernetes. On one hand, it’s genuinely the best tool we have for orchestrating containerized workloads at enterprise scale. Horizontal autoscaling, self-healing pods, rolling deployments, service discovery, all of that works well and I’d have a hard time replacing it.
On the other hand, the operational overhead is real. You need at least one or two people on the team who actually understand how the Kubernetes scheduler allocates pods, what happens when you set resource requests too low, how network policies work, and what to do when a persistent volume gets stuck in a terminating state. If nobody on your team has that knowledge, you’re going to have a bad time.
We always start with a managed service. EKS on AWS, AKS on Azure, GKE on Google Cloud. Running your own control plane is a waste of time unless you have a very specific compliance reason. Even with managed Kubernetes, though, there’s a lot to get right: namespace isolation between teams, resource quotas so one team’s runaway pod doesn’t starve everyone else, proper RBAC policies, and you need monitoring set up before your first real deployment. Not after the first outage.
One mistake I see constantly: teams set up autoscaling but never test it under load. They configure “scale up when CPU hits 70%” and assume it’ll work. Then Black Friday hits (or whatever their peak event is) and they discover that new pods take 45 seconds to start, and the load balancer takes another 30 seconds to register them as healthy, and in that 75-second gap their existing pods are getting crushed. Test your autoscaling. Do a load test. Please.
Multi-cloud: usually accidental, always painful
I don’t think I’ve ever met an enterprise that chose multi-cloud because they sat down and rationally decided it was the best architecture. It’s always something that happened to them. They acquired a company that runs on Azure while they run on AWS. Or their German subsidiary has to keep data in EU-based infrastructure for GDPR reasons and somebody picked Google Cloud for that. Or procurement negotiated Azure credits but engineering already built everything on AWS.
Whatever the reason, once you’re in a multi-cloud situation, you have to deal with it. Terraform helps a lot here because you can manage AWS, Azure, and GCP resources with the same tool and the same workflow. Container-based workloads are easier to move between clouds than VM-based ones, which is another reason to standardize on containers early.
My honest advice: don’t go multi-cloud on purpose. The “vendor lock-in” argument sounds good in a meeting, but the reality is that most lock-in comes from using managed services (RDS, DynamoDB, Cloud Functions), and you should be using those because they reduce how much stuff you have to operate yourself. The operational cost of running the same workload on two clouds is almost always higher than the hypothetical cost of being locked into one.
Security is a day-one problem
I’m not going to write a comprehensive guide to cloud security here because that’s its own series of posts. But there are a few things that come up on almost every enterprise project we work on.
First: zero-trust networking. The old approach of “everything inside the VPC is trusted” stopped making sense once you had more than a handful of services. Every service-to-service call should be authenticated. mTLS is the standard way to do this in Kubernetes (Istio and Linkerd both handle it). If you’re not on Kubernetes, at minimum use IAM roles and security groups to restrict what can talk to what.
IAM configuration is where most cloud security falls apart in practice. It starts fine, with carefully scoped roles and least-privilege access. Then six months in, someone can’t deploy because they’re missing a permission, so they get AdministratorAccess “temporarily,” and that temporary fix is still there two years later. We’ve started doing quarterly IAM audits on every project, just going through every role and asking “does this still need all these permissions?” It’s tedious but it catches a lot.
Compliance (SOC 2, ISO 27001, HIPAA, whatever applies to your industry) adds work, but it’s not as painful as people expect if you build for it from the start. AWS Config, Azure Policy, and GCP Organization Policies let you set guardrails that automatically flag or block non-compliant resources. The key is setting these up before you have 200 resources to retroactively audit.
Monitoring is half the job
I’m convinced that at least 50% of running reliable cloud infrastructure at enterprise scale is monitoring. Maybe more. You can have perfect architecture and still have outages if you don’t know what’s happening inside your system.
The industry has started saying “observability” instead of “monitoring,” and there is a real difference. Monitoring is “send me an alert when the CPU is above 90%.” Observability is “when a customer reports that checkout is slow, I can trace their request through 12 services and find out that the delay is in service #7 because it’s waiting on a database query that’s doing a sequential scan on a 50 million row table.” You need both.
We typically use Datadog or a Grafana/Prometheus stack for metrics and alerting. For distributed tracing, OpenTelemetry has become the standard and it’s worth instrumenting your services with it early. Adding tracing to an existing system with 30 services is a multi-month project that nobody wants to do. Adding it to each service as you build it takes an afternoon.
One thing I wish someone had told me earlier: your alerts are only useful if people pay attention to them. If your team gets 50 alerts a day and most of them are noise, they’ll start ignoring all of them, including the real ones. Spend the time to tune your alert thresholds. Delete alerts that nobody has acted on in the last month. An alert that fires and gets ignored is worse than no alert at all, because it creates a false sense of security.
Cloud costs will surprise you
Enterprise cloud bills have a way of getting out of control quietly. Nobody notices because the increments are small, and then someone in finance pulls a quarterly report and wants to know why you’re spending $180,000 a month on AWS when the budget was $120,000.
The biggest source of waste we see is over-provisioned resources. Dev and staging environments running 24/7 with production-sized instances. Services requesting 4 vCPUs and 16GB of RAM but actually using a fraction of that. Auto-scaling groups with minimums set too high because someone was nervous after a traffic spike last year.
Simple things help: tag every resource with the team and project that owns it. Set up a dashboard that shows cost per team per service. Turn off non-production environments outside business hours (or at least scale them way down). Use reserved instances or savings plans for your steady-state production workloads. Use spot instances for anything that can tolerate interruption (batch processing, CI/CD runners, data pipelines).
FinOps, the practice of making engineering teams aware of and accountable for their cloud spend, has been genuinely useful on our projects. When a team can see that their service costs $8,000 a month and most of it is an oversized RDS instance they could downsize, they usually fix it pretty quickly. When costs are just a line item in someone else’s budget, nobody cares.
Wrapping up
There’s no single right way to build scalable cloud infrastructure for enterprise applications. The specifics depend on your compliance requirements, your team’s skill set, your existing systems, and honestly, your budget. But the general principles we keep returning to after years of doing this work are: put your infrastructure in code, separate services that need to scale independently, invest in monitoring early and aggressively, take security seriously from day one, and keep an eye on costs before they become a crisis.
Oh, and test your auto-scaling under load. I can’t say that enough.