Is it best practice to use separate GCP projects for Firestore with microservices running in the same GKE cluster?

We're building a microservices backend that will be running in GKE.

Many of these microservices will use Firestore and I'm wondering about the best practice for GCP projects in this situation.

I'd like to have one cluster/GCP project per environment (dev, staging, prod), but since Firestore doesn't support multiple instances per project, we're planning to create additional projects for each service's Firestore DB in each environment (development-firestore-service1, staging-firestore-service2 etc), and put all related projects (main project with GKE cluster + Firestore projects) in a folder per environment.

As a DevOps guy I'm not too fond of this solution (cross-project IAM permissions will be a headache) and I'd like a simpler layout if possible.

I know there is scoping / namespacing at the Firestore level : is this considered sufficient isolation between "friendly" microservices (single tenant / team) and would it require significant refactoring work on our developers part ?

What is the best practice ?

Thanks

2 answers

  • answered 2021-06-18 19:33 Tarik Huber

    I would recommend to create a separate project for each environment.

    If you use only the firestore and not the cloud functions, hosting etc. you "could" use a single project and just add more databases for each environment. But as said I would always recommend to have separate projects. Even on the last Google IO on the Firebase live QA they recommended to have multiple projects for each env.

    You could create a REST API to sync data between the env. For example copy data from one to another and even users. You would need to write the code for it on your own but you would also have the control over it.

  • answered 2021-06-19 11:48 guillaume blaquiere

    I agree with Tarik. For a better Devops organisation and pipeline over the month and years, it's better to have fully isolated, and replicated environment.

    especially when you would like to try to update K8S or to use a new AddOn. If you crash your cluster, only the dev/test cluster is down, not the prod. If you have only one, you crash all!!

    however, I also saw in different company the wish to manage only one big cluster and to split it in namespace for each environment and project. The benefit is the single management of a big cluster (instead of several small cluster, very time consuming) and the cost optimization.

    In fact, it's like the discussion about Monorepo or not. There isn't only one good answer, it depends on your constraints and wishes.