Very slow performance after docker upgrade when using "localhost"

[DrewImm]

~10x slower after updating docker and rebuilding postgres containers -

I've been using docker postgres for a project which was very very fast for the past year or so until updating docker and rebuilding my postgres containers.

I'm still getting very fast performance for other docker database containers (mssql, mysql), confirming it's not an issue with docker.

I'm also getting very fast performance via postgres installed directly on Windows (without docker)

I tried reinstalling Windows and my entire dev environment, which did not resolve the issue.

I've also tried some solutions online such as shm_size etc., which also did not resolve the issue.

Application tests run in about 4 seconds on Windows-installed postgres:

And run in about 40 seconds on Docker-installed postgres:

• Docker Desktop 4.25.2 (129061)
• postgres:12, postgres:13, postgres:14, postgres:15

Docker compose:

docker compose up

version: '3.1'
services:
    # Postgres
    postgres-12:
        image: 'postgres:12'
        container_name: 'riao-dbal-postgres-12'
        ports:
            - '5432:5432'
        environment:
            POSTGRES_USER: 'riao_root'
            POSTGRES_PASSWORD: 'password1234'
            POSTGRES_DB: 'db'
        shm_size: '2gb'

    postgres-13:
        image: 'postgres:13'
        container_name: 'riao-dbal-postgres-13'
        ports:
            - '5433:5432'
        environment:
            POSTGRES_USER: 'riao_root'
            POSTGRES_PASSWORD: 'password1234'
            POSTGRES_DB: 'db'
        shm_size: '2gb'

    postgres-14:
        image: 'postgres:14'
        container_name: 'riao-dbal-postgres-14'
        ports:
            - '5434:5432'
        environment:
            POSTGRES_USER: 'riao_root'
            POSTGRES_PASSWORD: 'password1234'
            POSTGRES_DB: 'db'
        shm_size: '2gb'

    postgres-15:
        image: 'postgres:15'
        container_name: 'riao-dbal-postgres-15'
        ports:
            - '5435:5432'
        environment:
            POSTGRES_USER: 'riao_root'
            POSTGRES_PASSWORD: 'password1234'
            POSTGRES_DB: 'db'
        shm_size: '2gb'

Running command docker-compose.yml to build containers

Docker Inspect:

{
	"Id": "9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb",
	"Created": "2023-11-24T14:39:45.117368622Z",
	"Path": "docker-entrypoint.sh",
	"Args": [
		"postgres"
	],
	"State": {
		"Status": "running",
		"Running": true,
		"Paused": false,
		"Restarting": false,
		"OOMKilled": false,
		"Dead": false,
		"Pid": 19512,
		"ExitCode": 0,
		"Error": "",
		"StartedAt": "2023-11-24T14:42:12.63045035Z",
		"FinishedAt": "2023-11-24T14:42:03.891409372Z"
	},
	"Image": "sha256:8cde386e2e85cce0eb684d0b27ddf6a8abbecd05d5af43bbbdf7be12c47c44e8",
	"ResolvConfPath": "/var/lib/docker/containers/9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb/resolv.conf",
	"HostnamePath": "/var/lib/docker/containers/9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb/hostname",
	"HostsPath": "/var/lib/docker/containers/9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb/hosts",
	"LogPath": "/var/lib/docker/containers/9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb/9c2b71981d1c83ae2050274eb52c5a770cadc2af4cad0b6b0c1d1bc0a7e022eb-json.log",
	"Name": "/riao-dbal-postgres-15",
	"RestartCount": 0,
	"Driver": "overlay2",
	"Platform": "linux",
	"MountLabel": "",
	"ProcessLabel": "",
	"AppArmorProfile": "",
	"ExecIDs": null,
	"HostConfig": {
		"Binds": null,
		"ContainerIDFile": "",
		"LogConfig": {
			"Type": "json-file",
			"Config": {}
		},
		"NetworkMode": "riao-project_default",
		"PortBindings": {
			"5432/tcp": [
				{
					"HostIp": "",
					"HostPort": "5435"
				}
			]
		},
		"RestartPolicy": {
			"Name": "",
			"MaximumRetryCount": 0
		},
		"AutoRemove": false,
		"VolumeDriver": "",
		"VolumesFrom": null,
		"ConsoleSize": [
			0,
			0
		],
		"CapAdd": null,
		"CapDrop": null,
		"CgroupnsMode": "host",
		"Dns": null,
		"DnsOptions": null,
		"DnsSearch": null,
		"ExtraHosts": [],
		"GroupAdd": null,
		"IpcMode": "private",
		"Cgroup": "",
		"Links": null,
		"OomScoreAdj": 0,
		"PidMode": "",
		"Privileged": false,
		"PublishAllPorts": false,
		"ReadonlyRootfs": false,
		"SecurityOpt": null,
		"UTSMode": "",
		"UsernsMode": "",
		"ShmSize": 2147483648,
		"Runtime": "runc",
		"Isolation": "",
		"CpuShares": 0,
		"Memory": 0,
		"NanoCpus": 0,
		"CgroupParent": "",
		"BlkioWeight": 0,
		"BlkioWeightDevice": null,
		"BlkioDeviceReadBps": null,
		"BlkioDeviceWriteBps": null,
		"BlkioDeviceReadIOps": null,
		"BlkioDeviceWriteIOps": null,
		"CpuPeriod": 0,
		"CpuQuota": 0,
		"CpuRealtimePeriod": 0,
		"CpuRealtimeRuntime": 0,
		"CpusetCpus": "",
		"CpusetMems": "",
		"Devices": null,
		"DeviceCgroupRules": null,
		"DeviceRequests": null,
		"MemoryReservation": 0,
		"MemorySwap": 0,
		"MemorySwappiness": null,
		"OomKillDisable": false,
		"PidsLimit": null,
		"Ulimits": null,
		"CpuCount": 0,
		"CpuPercent": 0,
		"IOMaximumIOps": 0,
		"IOMaximumBandwidth": 0,
		"MaskedPaths": [
			"/proc/asound",
			"/proc/acpi",
			"/proc/kcore",
			"/proc/keys",
			"/proc/latency_stats",
			"/proc/timer_list",
			"/proc/timer_stats",
			"/proc/sched_debug",
			"/proc/scsi",
			"/sys/firmware"
		],
		"ReadonlyPaths": [
			"/proc/bus",
			"/proc/fs",
			"/proc/irq",
			"/proc/sys",
			"/proc/sysrq-trigger"
		]
	},
	"GraphDriver": {
		"Data": {
			"LowerDir": "/var/lib/docker/overlay2/2280e4453b2f806888f7b463c0d48cf3049633e12ae237dcdf6d7766a20e7b64-init/diff:/var/lib/docker/overlay2/24d04924c42a0c62aa8dd43d006268bbc9a301293a82717bffa4c865925cd42c/diff:/var/lib/docker/overlay2/ea3dbdc4d33399fd1d1b98a91cad88edf2bd8adfaa37bf6de592b6c0e9ccfcf5/diff:/var/lib/docker/overlay2/83e093cc54287e72590765cf94cf02c56fc9a9b2e1e5a7df90b2e7b08ea0d8eb/diff:/var/lib/docker/overlay2/ae4fec3031864d4488811bd3324ee20ffb2cfe56644409c406273068fd9eaa1b/diff:/var/lib/docker/overlay2/877ab6bcf4d032ca3dcf772cba6951c075588e659c4e4da8d5b3e9998048ac5d/diff:/var/lib/docker/overlay2/5e422fd6e58a93f326762b519229b6e246545a56bdf5b580ea778fa994166a91/diff:/var/lib/docker/overlay2/39bcb7e82105e2755f35df966fc67694de39a3e22fdb1fc4d8c39fee4a31e14f/diff:/var/lib/docker/overlay2/7a6b26948b50b212dd301883cdee82f3f8e90b07a57fcf107dbe8f2809742cdc/diff:/var/lib/docker/overlay2/230fd2e7ac49edc415f0d58b4b2defbac3de894f1f23c5fcc42fe5eb5ad14e05/diff:/var/lib/docker/overlay2/caef1238bb1bb29a6c3a26dd1df9444c5de56768d9d8b19edc4af7f04c93c4a9/diff:/var/lib/docker/overlay2/a5727870ce2126f6f2cc5f4d2f4edfec25b738f3868232aa93b512a10191fe7b/diff:/var/lib/docker/overlay2/3243cd85857457a3136085ebd4b6c6522a73e90177887aa8d1821f1a60bf6a75/diff:/var/lib/docker/overlay2/eed2632f98f36b4bd829bfcca2837866ff38a5a938c009c8adde8bc336dbd09c/diff",
			"MergedDir": "/var/lib/docker/overlay2/2280e4453b2f806888f7b463c0d48cf3049633e12ae237dcdf6d7766a20e7b64/merged",
			"UpperDir": "/var/lib/docker/overlay2/2280e4453b2f806888f7b463c0d48cf3049633e12ae237dcdf6d7766a20e7b64/diff",
			"WorkDir": "/var/lib/docker/overlay2/2280e4453b2f806888f7b463c0d48cf3049633e12ae237dcdf6d7766a20e7b64/work"
		},
		"Name": "overlay2"
	},
	"Mounts": [
		{
			"Type": "volume",
			"Name": "7b2909b6e24d0b48399c3137ba14ab3fe679e6e85423c3fdfd501a5e57eba8f9",
			"Source": "/var/lib/docker/volumes/7b2909b6e24d0b48399c3137ba14ab3fe679e6e85423c3fdfd501a5e57eba8f9/_data",
			"Destination": "/var/lib/postgresql/data",
			"Driver": "local",
			"Mode": "",
			"RW": true,
			"Propagation": ""
		}
	],
	"Config": {
		"Hostname": "9c2b71981d1c",
		"Domainname": "",
		"User": "",
		"AttachStdin": false,
		"AttachStdout": true,
		"AttachStderr": true,
		"ExposedPorts": {
			"5432/tcp": {}
		},
		"Tty": false,
		"OpenStdin": false,
		"StdinOnce": false,
		"Env": [
			"POSTGRES_DB=db",
			"POSTGRES_USER=riao_root",
			"POSTGRES_PASSWORD=password1234",
			"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/15/bin",
			"GOSU_VERSION=1.16",
			"LANG=en_US.utf8",
			"PG_MAJOR=15",
			"PG_VERSION=15.5-1.pgdg120+1",
			"PGDATA=/var/lib/postgresql/data"
		],
		"Cmd": [
			"postgres"
		],
		"Image": "postgres:15",
		"Volumes": {
			"/var/lib/postgresql/data": {}
		},
		"WorkingDir": "",
		"Entrypoint": [
			"docker-entrypoint.sh"
		],
		"OnBuild": null,
		"Labels": {
			"com.docker.compose.config-hash": "9b63b255858e032200e43aedfe79db5746b18ca786c4a1596b06129e911bedae",
			"com.docker.compose.container-number": "1",
			"com.docker.compose.depends_on": "",
			"com.docker.compose.image": "sha256:8cde386e2e85cce0eb684d0b27ddf6a8abbecd05d5af43bbbdf7be12c47c44e8",
			"com.docker.compose.oneoff": "False",
			"com.docker.compose.project": "riao-project",
			"com.docker.compose.project.config_files": "D:\\Dev\\riao-project\\docker-compose.yml",
			"com.docker.compose.project.working_dir": "D:\\Dev\\riao-project",
			"com.docker.compose.service": "postgres-15",
			"com.docker.compose.version": "2.23.0"
		},
		"StopSignal": "SIGINT"
	},
	"NetworkSettings": {
		"Bridge": "",
		"SandboxID": "c25224fa93ce51fb51852e2da0c9a9f7b9b419d6e74e59e252152adcbe17694c",
		"HairpinMode": false,
		"LinkLocalIPv6Address": "",
		"LinkLocalIPv6PrefixLen": 0,
		"Ports": {
			"5432/tcp": [
				{
					"HostIp": "0.0.0.0",
					"HostPort": "5435"
				}
			]
		},
		"SandboxKey": "/var/run/docker/netns/c25224fa93ce",
		"SecondaryIPAddresses": null,
		"SecondaryIPv6Addresses": null,
		"EndpointID": "",
		"Gateway": "",
		"GlobalIPv6Address": "",
		"GlobalIPv6PrefixLen": 0,
		"IPAddress": "",
		"IPPrefixLen": 0,
		"IPv6Gateway": "",
		"MacAddress": "",
		"Networks": {
			"riao-project_default": {
				"IPAMConfig": null,
				"Links": null,
				"Aliases": [
					"riao-dbal-postgres-15",
					"postgres-15",
					"9c2b71981d1c"
				],
				"NetworkID": "7d0007ec201ae063e353a7b099f77c7112e3f1e4639d774b4f952a2d9060a6cc",
				"EndpointID": "fa8e7ab9e4b4cda2f619e5d3682b77b8a80b5f12da1aec459425bcd7bc154982",
				"Gateway": "172.23.0.1",
				"IPAddress": "172.23.0.2",
				"IPPrefixLen": 16,
				"IPv6Gateway": "",
				"GlobalIPv6Address": "",
				"GlobalIPv6PrefixLen": 0,
				"MacAddress": "02:42:ac:17:00:02",
				"DriverOpts": null
			}
		}
	}
}

[DrewImm]

Solved w/ workaround - I changed the client's host variable from localhost to 0.0.0.0...

• Not sure why this affects postgres now but was fine 7 months ago when docker-postgres was first installed
• Not sure why this affects docker-postgres but not docker-mssql, docker-mysql
• Not sure why this affects docker-postgres but not native Windows postgres installation

Thanks!

[KujanenJPareto]

Experienced the same issue when upgrading windows docker desktop. Changed localhost to 0.0.0.0. Solved the issue.

[ChaosO9]

@KujanenJPareto

Experienced the same issue when upgrading windows docker desktop. Changed localhost to 0.0.0.0. Solved the issue.

How do I do that?

[KujanenJPareto]

Depends on how you are connecting to the instance. I was using knex, so i had a config that looks something like:

const knexConfig: Knex.Config = {
  client: 'pg',
  connection: async () => {
    const connectionConfig = {
      user: process.env.DB_USER,
      port: process.env.DB_PORT
      host: process.env.DB_HOST,
    };
    return connectionConfig;
  },
};

And then instead of having
DB_HOST=localhost in my env, i had DB_HOST=0.0.0.0

[Parashoe]

This remains unresolved and very puzzling.

Using Windows with Docker Desktop and postgres:lastest: I'm getting around 50ms+ of latency on top of the query execution time when using localhost but it's normal (around 1-5ms on top of execution time) with 0.0.0.0 after the first query.

It's almost like connecting with localhost is treating every query like the first? A guess.

[MattParkerDev]

I'm seeing this too. My surface level guess is that it is to do with DNS resolution of localhost (not necessary when using 127.0.0.1 or 0.0.0.0)