Before diving into instance sizes and costs, let’s first break down what an instance is and why it’s so important for deploying your data stack.
An instance is essentially a virtual machine that runs in the cloud. Think of it as a remote computer or server that you rent to run your software and tools. For every tool or service in your data stack, you'll need an instance to run it on. The instance provides the computing power and storage capacity for these tools to function properly.
When you’re setting up a data stack, you’re bringing together multiple tools for things like data ingestion, transformation, orchestration, and visualization. These tools need a powerful environment to operate, and that’s where instances come into play. Without an instance, there’s no place for your tools to live and run. It’s like trying to run a complex software program without having a computer—it just doesn’t work!
Each instance allows you to:
Instances have a cost because they use physical resources from cloud providers like Google Cloud. You are essentially renting CPU power, memory, and storage. The larger or more powerful the instance, the more it costs. This cost is passed down based on the machine size you select (e.g., XS, S, M, L, XL), how many resources it uses, and how long it’s running.
While Visionarist simplifies and automates much of this process, it's important to understand that deploying and running your data stack requires real computing power, which is why there’s a cost involved when you scale up your operations.