(Terraform) Error 400: Invalid request: instance name (pg_instance)., invalid
On GCP, I'm trying to create a Cloud SQL instance with this Terraform code below:
resource "google_sql_database_instance" "postgres" {
name = "pg_instance"
database_version = "POSTGRES_13"
region = "asia-northeast1"
deletion_protection = false
settings {
tier = "db-f1-micro"
disk_size = 10
}
}
resource "google_sql_user" "users" {
name = "postgres"
instance = google_sql_database_instance.postgres.name
password = "admin"
}
But I got this error:
Error: Error, failed to create instance pg_instance: googleapi: Error 400: Invalid request: instance name (pg_instance)., invalid
Are there any mistakes for my Terraform code?
1 answer
-
answered 2022-05-07 01:31
Kai - Kazuya Ito
For a Cloud SQL instance name, only lowercase letters, numbers, and hyphens are allowed and it must start with a letter.
So, change this:
name = "pg_instance" # Underscore "_" is not allowed
To this:
name = "pg-instance" # Hyphen "-" is allowed
In addition, on GUI, you can see this message below under the blank for instance ID("name" in Terraform):
Use lowercase letters, numbers, and hyphens. Start with a letter.
do you know?
how many words do you know
See also questions close to this topic
-
Google Cloud Compute Engine http Connection Timeout
I have setup a compute engine VM with 2vCPU and 2GB RAM.I have setup nginx server and setup the firewalls permissions as shown in the diagram. When I try to access the angular files hosted on the server using the external IP I get the error "The connection has timed out" and when I try to use curl on the terminal, it displays the error "curl: (28) Failed to connect to IP port 80 after 129163 ms: Connection timed out".
Both the Http and Https firewall rules are enabled
Whe I run the command
sudo systemctl status apache2
netstat -tulpn | grep LISTEN
enter code here
Any ideas on what the issue might be will be really helpful
-
Apache beam FixedWindow doesn't do anything after GroupByKey transform
I built a pipeline which reads from confluent kafka it processes the records and then use side outputs to split them into rejected and approved pcollections, then the approved pcollections gets written to bigquery, but I want to persist the approved records and write them into a file on gcs.
The code is:
windowing=(aproved | 'Create_window' >> beam.WindowInto(window.FixedWindows(60)) | 'AddKey' >> beam.Map(lambda record: (none,record)) | 'GBK' >> beam.GroupByKey() | 'remove_key' >> beam.FlatMap(ret_key) | 'AddTimeStamp' >> beam.Map(lambda record: beam.window.TimestampValue(record,time.time())) | 'Write' >> WriteToFiles(path=MY_BUCKET,file_naming=destination_prefix_naming('.ppl')) )
This works when I test it reading from a file and using direct runner, but when I use dataflow and streaming it just doesn't do anything after the GroupByKey transform, it says on the graph that 20 element were added, but the next transform ('remove_key') never gets an element after that
-
Context root redirect for AWS Application Load Balancer using terraform
I have an AWS ALB that I need to add a context-root redirect but I can not figure out the syntax in Terraform.
I want this url to automatically go to this url. I know it gets added to the https listener rule, but I just do not know how to syntactically write it
example.com --> example.com/mycode
This current iteration of code complains with But I do not even know if this syntax is correct:
Error: Unsupported argument on main.tf line 133, in module "alb": 133: conditions = [{ An argument named "conditions" is not expected here.
terraform plan operation failed this is the code I have:
https_listener_rules = [ { action_type = "redirect" redirect = { host = "example.com" path = "/mycode" port = 443 protocol = "HTTPS" status_code = "HTTP_302" } } ] conditions = [{ path_patterns = ["/"] }]
-
How does Kubernetes and Terraform work seamlessly together and what role do they each undertake?
I am a bit confused about the individual roles of Kubernetes and Terraform when using them both on a project.
Until very recently, I had a very clear understanding of both their purposes and everything made sense to me. But, then I heard in one of Nana's videos on Terraform, that Terraform was also very advanced in orchestration and I got confused.
Here's my current understanding of both these tools:
Kubernetes: Orchestration software that controls many docker containers working together seamlessly. Kubernetes makes sure that new containers are deployed based on the desired infrastructure defined in configuration files (written with the help of a tool like Terraform, as IaC).
Terraform: Tool for provisioning, configuring, and managing infrastructure as IaC.
So, when we say that Terraform is a good tool for orchestration, do we mean that it's a good tool for orchestrating infrastructure states or docker containers as well?
I hope someone can clear that out for me!
-
Automate Azure Devops (FTP Upload) and Git to upload on Remote Server
The current setup is as below
- Version Control - Git
- Repos and Branch hosted on - Azure DevOps
- Codebase - External server
The dev team clones Azure Repo into local git project and any staged changes are committed via Git and pushed to specific branch of Azure DevOps. In this setup we would want to upload the changes to external FTP servers and avoid manual upload. Currently trying to use Azure Devops FTP Upload Task (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/ftp-upload?view=azure-devops), however facing issues; yaml script as below
trigger: - main pool: vmImage: 'ubuntu-latest' variables: phpVersion: 7.4 webAppName: 'Test Project' buildConfiguration: 'Release' vmImageName: 'ubuntu-latest' steps: - publish: $(System.DefaultWorkingDirectory)/AzureRepoName artifact: Test Project Deploy - task: FtpUpload@2 displayName: 'FTP Upload' inputs: credentialsOption: inputs serverUrl: 'ftps://00.00.00.00:22' username: ftp-username password: ftp-password rootDirectory: '$(System.DefaultWorkingDirectory)/AzureRepoName' remoteDirectory: '/home/public_html' clean: false cleanContents: false preservePaths: true trustSSL: true
PROBLEM
Following errors occur when I commit (for test purposes) something.
Starting: PublishPipelineArtifact ============================================================================== Task : Publish Pipeline Artifacts Description : Publish (upload) a file or directory as a named artifact for the current run Version : 1.199.0 Author : Microsoft Corporation Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/publish-pipeline-artifact ============================================================================== Artifact name input: Test Project Deploy ##[error]Path does not exist: /home/vsts/work/1/s/AzureRepoName Finishing: PublishPipelineArtifact
I want to upload any staged change that is committed to main branch on Azure Devops to be automatically deploy on the remote FTP server
Thanks
-
I want to export a single database and import it in another instance
We've multiple databases under 1 mysql instance on gcp. I want to export a single database (that should include SP, Triggers, views, tables, data) from gcp mysql and import it in another instance. My database would have 100gb of data e.g, i tried gcloud export but that does not import SP. mysqldump would be the option for the same but not sure how to use it to export it in a bucket and import it as new database. Can you please help me here?
-
Cant connect to Cloud SQL from Cloud Composer using Public IP
I'm trying to connect my instance of Google Composer to Google cloud Postgres. I am trying to do this via the Public IP. The DAG error is saying I cant connect to the server. The DAG is using a postgres operator.
I then have a connection configured in my composer instance. This has the public ip of my Cloud SQL Postgres server.
I think the error is around not allowing the cloud composer ip into the firewall for the Cloud SQL server. Am I doing this the wrong way? Should I be using a different operator? Ive been stuck on this for a few days now and exhausted my options from google.
Any help would be greatly appreciated!
Thanks
-
How to deal with django migration postgres deadlock?
So, I was deploying the django app to production and then the infamous postgres deadlock situation happened. This happens during the migration.
Django version: 3.2 Postgres 13 Google cloud sql postgres.
OperationalError deadlock detected DETAIL: Process 112506 waits for AccessExclusiveLock on relation 29425 of database 27145; blocked by process 112926. Process 112926 waits for RowShareLock on relation 29381 of database 27145; blocked by process 112506. HINT: See server log for query details.
I ran this query to get the process info:
SELECT 29425::regclass,29381::regclass from pg_locks;
and result:
regclass | regclass "custom_requestlog" "auth_user" "custom_requestlog" "auth_user"
I am not sure how to proceed ahead, as the pgaudit has been enabled but it doesnt show anything and also the query insights is not that helpful. Attached is image of query insights.
Any help would be helpful please!
Update: The query from log explorer in Google cloud, gave this query just after deadlock detected error:
2022-04-29 13:51:36.590 UTC [6445]: [732-1] db=xyz_prod,user=backend-prod DETAIL: Process 6445 waits for AccessExclusiveLock on relation 29425 of database 27145; blocked by process 9249. Process 9249 waits for RowShareLock on relation 29381 of database 27145; blocked by process 6445. Process 6445: SET CONSTRAINTS "custom_requestlog_user_id_3ff3f1cf_fk_some_user_id" IMMEDIATE; ALTER TABLE "custom_requestlog" DROP CONSTRAINT "custom_requestlog_user_id_3ff3f1cf_fk_some_user_id" Process 9249: INSERT INTO "custom_requestlog" ("user_id", "ip_addr", "url", "session_key", "method", "headers", "query", "body", "cookies", "timestamp", "status_code", "response_snippet") VALUES (NULL, 'xx.xxx.xx.xxx'::inet, '/version/', NULL, 'GET', '{"HTTP_HOST": "api.some.com", "HTTP_ACCEPT": "*/*", "HTTP_ACCEPT_ENCODING": "deflate, gzip", "HTTP_USER_AGENT": "GoogleStackdriverMonitoring-UptimeChecks(https://cloud.google.com/monitoring)", "HTTP_X_CLOUD_TRACE_CONTEXT": "xxxxxx/9771676669485105781", "HTTP_VIA": "1.1 google", "HTTP_X_FORWARDED_FOR": "xx.xxx.xx.xxx, xx.xxx.xx.xxx", "HTTP_X_FORWARDED_PROTO": "https", "HTTP_CONNECTION": "Keep-Alive"}', '{}', '\x'::bytea, '{}', '2022-04-29T13:48:46.844830+00:00'::timestamptz, 200, NULL) RETURNING "custom_requestlog"."id"
-
Problem with creating compute instance in GCP for different project IDs with terraform
I have a terraform to create compute instance in GCP which looks like:
resource "google_compute_address" "static_ip" { project = var.project_id name = "vm-instance" } resource "google_compute_instance" "vm_instance" { project = var.project_id name = "vm-instance" machine_type = "e2-standard-4" zone = "us-east4-c" boot_disk { device_name = "vm-instance" mode = "READ_WRITE" initialize_params { image = "centos-cloud/centos-7" size = 20 } } tags = ["web-host","http-server","https-server"] network_interface { network = var.network subnetwork = var.subnetwork #subnetwork_project = var.project_id access_config { nat_ip = google_compute_address.static_ip.address } } service_account { scopes = ["cloud-platform"] } scheduling { on_host_maintenance = "MIGRATE" #provisioning_model = "STANDARD" } shielded_instance_config { enable_secure_boot = true enable_vtpm = true enable_integrity_monitoring = true } }
and modules which will use it:
module "sandbox_vm" { source = "./modules/terraform-vm" network = "sandbox-vpc" subnetwork = "sandbox-vpc" project_id = var.sandbox_project } module "dev_vm" { source = "./modules/terraform-vm" network = "dev-vpc" subnetwork = "dev-vpc" project_id = var.dev_project } module "prod_vm" { source = "./modules/terraform-vm" network = "prod-vpc" subnetwork = "prod-vpc" project_id = var.prod_project }
but when I will try to apply it will end up with errors:
Error: Error creating instance: googleapi: Error 400: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'projects/cloud-sandbox/regions/us-east4/subnetworks/dev-vpc'. The referenced subnetwork resource cannot be found., invalid on modules/terraform-vm/main.tf line 6, in resource "google_compute_instance" "vm_instance": 6: resource "google_compute_instance" "vm_instance" { Error: Error creating instance: googleapi: Error 400: Invalid value for field 'resource.networkInterfaces[0].subnetwork': 'projects/cloud-sandbox/regions/us-east4/subnetworks/prod-vpc'. The referenced subnetwork resource cannot be found., invalid on modules/terraform-vm/main.tf line 6, in resource "google_compute_instance" "vm_instance": 6: resource "google_compute_instance" "vm_instance" {
Why it will try to use subnets from other project IDs when those are strictly specified per module?
PS.
Provider file contains
provider "google" { credentials = var.credentials_file region = var.region }
-
How to use "google_service_account_access_token" scopes
I am trying to set-up my terraform's
main.tf
by using service accounts that get impersonated by authorized users... I followed this guide to set this up, and it works, I can create a bucket within my project and within my organization.Now in order to do more specific stuff, like declare these blocks
data "google_organization" "org" { organization = var.organization.id } data "google_billing_account" "acct" { billing_account = var.billing_account.id open = var.billing_account.active }
I figured (but I may be wrong) that I need to include more elements in the scope of my provider, like this :
provider "google" { alias = "super_admin_impersonation" scopes = [ "https://www.googleapis.com/auth/cloud-platform", "https://www.googleapis.com/auth/userinfo.email", "https://www.googleapis.com/auth/admin.directory.orgunit", # I added this "https://www.googleapis.com/auth/cloud-billing" # And this ] }
and so, also add them to the
google_service_account_access_token
blockdata "google_service_account_access_token" "super_admin" { provider = google.super_admin_impersonation target_service_account = "${var.service_acc_terraform_super_admin.name}@${var.project_infra_genesis.id}.${var.service_acc_terraform_super_admin.suffix}" scopes = ["cloud-platform", "userinfo-email", "admin.directory.orgunit", "cloud-billing"] lifetime = "1200s" }
The problem, is that when I add
"admin.directory.orgunit", "cloud-billing"
to this block's scope, I get this error :╷ │ Error: googleapi: Error 400: Request contains an invalid argument., badRequest │ │ with data.google_service_account_access_token.super_admin, │ on main.tf line 22, in data "google_service_account_access_token" "super_admin": │ 22: data "google_service_account_access_token" "super_admin" { │ ╵
Google's official Oauth2 scopes for their APIs is where I determine the names of scopes...
My question is: how do you know which scopes can be used in
google_service_account_access_token
? Is there a complete list I can refer to in order to include more in this block? -
A managed resource has not been declared in module
I've a two modules under modules directory named as prienv and vpc
nnice@MPL-G8WW7D3:~/terra-test/modules/prienv$ pwd /home/nnice/terra-test/modules/prienv nnice@MPL-G8WW7D3:~/terra-test/modules/prienv$ ls data.tf main.tf variable.tf
Here is my data.tf file under prienv module
nnice@MPL-G8WW7D3:~/terra-test/modules/prienv$ cat data.tf data "aws_security_groups" "mysg-data" { filter { name = "tag:Name" values = ["MySG"] } depends_on = [aws_security_groups.mysg] } data "aws_subnet" "myprisn-data" { filter { name = "tag:Name" values = ["MyPriSN"] } depends_on = [aws_subnet.myprisn] }
Here is my variable.tf file under prienv module
variable "pri_av" { description = "for private availability zone" type = string default = "us-east-1b" } variable "ami" { description = "ami for ec2" type = string default = "ami-03ededff12e34e59e" } variable "instance_type" { description = "Instance type t2.micro" type = string default = "t2.micro" #will use double quotes for string type } variable "key_pair" { description = "key pair for ec2" type = string default = "learner_key" } variable "instance_count" { description = "EC2 instance count" type = number default = 1 #here we're using it to create two private instances } variable "project_environment" { description = "project name and environment for private instances" type = map(string) default = { project = "private", # it works upon key value pair where project is for key and environment is value environment = "testing" # you will find them under tag on aws console } }
And this is my main.tf file under prienv module
resource "aws_instance" "mypriec2" { ami = var.ami instance_type = var.instance_type count = var.instance_count availability_zone = var.pri_av #subnet_id = aws_subnet.myprisn.id subnet_id = data.aws_subnet.myprisn-data.id #vpc_security_group_ids = [aws_security_group.mysg.id] vpc_security_group_ids = data.aws_security_groups.mysg-data.ids key_name = var.key_pair # disable_api_termination = true tags = var.project_environment } resource "aws_key_pair" "mykeypair" { key_name = var.key_pair public_key = "ssh-rsa AAAAB3NaffrWscf59juCakElys9F3+zVuz0ta4gRUtKgWVPIj6ACr00VNDzsTTW2/sSjYtE5zWolVKCITlhqiIhgRKUDLKoxclxUKnK6IGIafdaefafaheiufa;fdaeoasfdkQvNtGrrHzY5/dbZhIUTxDUyvT5O5U= nnice@MPL-G8WW7D3" }
and here is my vpc moduls
nnice@MPL-G8WW7D3:~/terra-test/modules/vpc$ pwd /home/nnice/terra-test/modules/vpc nnice@MPL-G8WW7D3:~/terra-test/modules/vpc$ ls data.tf main.tf variable.tf
This is my data.tf file under vpc module
nnice@MPL-G8WW7D3:~/terra-test/modules/vpc$ cat data.tf data "aws_vpc" "myvpc-data" { filter { name = "tag:Name" values = ["MyVPC"] } depends_on = [aws_vpc.myvpc] } data "aws_subnet" "mypubsn-data" { filter { name = "tag:Name" values = ["MyPubSN"] } depends_on = [aws_subnet.mypubsn] } data "aws_subnet" "myprisn-data" { filter { name = "tag:Name" values = ["MyPriSN"] } depends_on = [aws_subnet.myprisn] }
This is my main.tf file under vpc module
nnice@MPL-G8WW7D3:~/terra-test/modules/vpc$ cat main.tf ################################################################## ############################## VPC ############################### ################################################################## resource "aws_vpc" "myvpc" { cidr_block = var.vpc_cidr instance_tenancy = var.vpc_tenancy tags = { Name = var.vpc_tag } } ################################################################## ############################# Subnet ############################# ################################################################## #PUBLIC SUBNET resource "aws_subnet" "mypubsn" { #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id cidr_block = var.mypubsn_cidr availability_zone = var.pub_av map_public_ip_on_launch = var.map_public_ip_on_launch tags = { Name = var.mypubsn_tag } } #PRIVATE SUBNET resource "aws_subnet" "myprisn" { #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id cidr_block = var.myprisn_cidr availability_zone = var.pri_av tags = { Name = var.myprisn_tag } } ################################################################## ############################### IGW ############################## ################################################################## resource "aws_internet_gateway" "myigw" { #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id tags = { Name = var.igw_tag } } ################################################################## ############################ Route Table ######################### ################################################################## #PUBLIC RT resource "aws_route_table" "mypubrt" { #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id tags = { Name = var.mypubsn_tag } } #PRIVATE RT resource "aws_route_table" "myprirt" { #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id tags = { Name = var.myprisn_tag } } #################################################################### ######################## Route Table Associate ##################### #################################################################### #PUBLIC RT association resource "aws_route_table_association" "pub" { #subnet_id = aws_subnet.mypubsn.id subnet_id = data.aws_subnet.mypubsn-data.id route_table_id = aws_route_table.mypubrt.id } #PRIVATE RT association resource "aws_route_table_association" "pri" { #subnet_id = aws_subnet.myprisn.id subnet_id = data.aws_subnet.myprisn-data.id route_table_id = aws_route_table.myprirt.id } ################################################################### ########################### Route ################################# ################################################################### #PUBLIC Route resource "aws_route" "mypubroute" { route_table_id = aws_route_table.mypubrt.id destination_cidr_block = var.pubroute gateway_id = aws_internet_gateway.myigw.id depends_on = [aws_route_table.mypubrt] } #PRIVATE Route #resource "aws_route" "mypriroute" { # route_table_id = aws_route_table.myprirt.id # destination_cidr_block = "0.0.0.0/0" # gateway_id = aws_internet_gateway.myigw.id # depends_on = [aws_route_table.myprirt] #} ################################################################### ############################ SG ################################### ################################################################### resource "aws_security_group" "mysg" { name = "MySecurityGroup" description = "Allow TLS inbound traffic" #vpc_id = aws_vpc.myvpc.id vpc_id = data.aws_vpc.myvpc-data.id ingress { description = "TLS from VPC" from_port = 22 to_port = 22 protocol = "tcp" # cidr_blocks = [aws_vpc.myvpc.cidr_block] cidr_blocks = ["0.0.0.0/0"] # ipv6_cidr_blocks = [aws_vpc.main.ipv6_cidr_block] } ingress { description = "TLS from VPC" from_port = 80 to_port = 80 protocol = "tcp" # cidr_blocks = [aws_vpc.myvpc.cidr_block] cidr_blocks = ["0.0.0.0/0"] # ipv6_cidr_blocks = [aws_vpc.main.ipv6_cidr_block] } egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } tags = { Name = var.SG_tag } }
And this is my variable.tf file under vpc module
nnice@MPL-G8WW7D3:~/terra-test/modules/vpc$ cat variable.tf variable "vpc_tenancy" { description = "vpc instance tenancy" type = string default = "default" } variable "pub_av" { description = "for public availability zone" type = string default = "us-east-1a" } variable "pri_av" { description = "for private availability zone" type = string default = "us-east-1b" } variable "vpc_tag" { description = "Tag for VPC" type = string default = "MyVPC" } variable "vpc_cidr" { description = "for vpc cidr" type = string default = "10.0.0.0/16" } variable "mypubsn_cidr" { description = "for public subnet cidr" type = string default = "10.0.1.0/24" } variable "myprisn_cidr" { description = "for private subnet cidr" type = string default = "10.0.2.0/24" } variable "mypubsn_tag" { description = "tag for public subnet" type = string default = "MyPubSN" } variable "myprisn_tag" { description = "tag for private subnet" type = string default = "MyPriSN" } variable "igw_tag" { description = "tag for IGW subnet" type = string default = "MyIGW" } variable "pubrt_tag" { description = "tag for private subnet" type = string default = "MyPubRT" } variable "prirt_tag" { description = "tag for IGW subnet" type = string default = "MyPriRT" } variable "pubroute" { description = "cidr for public route" type = string default = "0.0.0.0/0" } variable "SG_tag" { description = "tag for SG" type = string default = "MySG" } variable "map_public_ip_on_launch" { description = "auto enable public ip to public subnet" type = bool default = true }
And there is env directory where I have my main.tf file
nnice@MPL-G8WW7D3:~/terra-test$ ls env modules nnice@MPL-G8WW7D3:~/terra-test$ cd env/private-ec2/ nnice@MPL-G8WW7D3:~/terra-test/env/private-ec2$ pwd /home/nnice/terra-test/env/private-ec2 nnice@MPL-G8WW7D3:~/terra-test/env/private-ec2$ ls main.tf nnice@MPL-G8WW7D3:~/terra-test/env/private-ec2$ cat main.tf #Provider provider "aws" { region = "us-east-1" } #VPC module "vpc" { source = "../../modules/vpc" } #EC2 module "prienv" { source = "../../modules/prienv" }
When I'm trying to run terraform plan, I'm getting following errors
Error: Reference to undeclared resource │ │ on ../../modules/prienv/data.tf line 6, in data "aws_security_groups" "mysg-data": │ 6: depends_on = [aws_security_groups.mysg] │ │ A managed resource "aws_security_groups" "mysg" has not been declared in module.prienv. ╵ ╷ │ Error: Reference to undeclared resource │ │ on ../../modules/prienv/data.tf line 14, in data "aws_subnet" "myprisn-data": │ 14: depends_on = [aws_subnet.myprisn] │ │ A managed resource "aws_subnet" "myprisn" has not been declared in module.prienv.
Could anyone please let me know its solution? I already using vpc module in my main.tf file