problem
CreatePortForwardingRuleCmd on a VPC tier doesn't trigger ipassoc ( firewall and conserve mode on in vpc offering)
versions
ACS 4.23
The steps to reproduce the bug
4.23 introduced the conserve mode and firewall feature in vpc
https://docs.cloudstack.apache.org/en/4.23.0.0/adminguide/networking/virtual_private_cloud_config.html
- Create a vpc offering with firewall service and conserve mode enabled
- Launch a vpc network with the vpc offering
- Acquire a public ip
- Create a firewall rule
- Create a tier and launch a vm in the tier
- Create a PF rule on the same public ip to the vm
- Try to ssh to the vm on port 22 > ssh fails
- Login to the router and found that the public ip is not associated with the eth1 interface
root@r-184-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 0e:00:a9:fe:42:03 brd ff:ff:ff:ff:ff:ff
altname enp0s3
altname ens3
inet 169.254.66.3/16 brd 169.254.255.255 scope global eth0
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 1e:01:53:00:00:16 brd ff:ff:ff:ff:ff:ff
altname enp0s8
altname ens8
inet 192.168.55.83/24 brd 192.168.55.255 scope global eth1
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 02:01:00:fb:00:02 brd ff:ff:ff:ff:ff:ff
altname enp0s9
altname ens9
inet 172.30.45.2/28 brd 172.30.45.15 scope global eth2
valid_lft forever preferred_lft forever
logs
Management server log
root@Cloudstack-423:/home/ubuntu# cat /var/log/cloudstack/management/management-server.log |grep -i "logid:a308b23b"
2026-09-10 15:11:12,791 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-106:[ctx-1340fe97, job-1600]) (logid:a308b23b) Executing AsyncJob {"accountId":2,"cmd":"org.apache.cloudstack.api.command.user.firewall.CreatePortForwardingRuleCmd","cmdInfo":"{\"openfirewall\":\"false\",\"publicendport\":\"22\",\"ipaddressid\":\"230299bc-723e-4b8d-8e78-02073099fade\",\"sessionkey\":\"b6TLmuWlwTcmSFzpYdqibd9ne-o\",\"httpmethod\":\"POST\",\"privateendport\":\"22\",\"ctxAccountId\":\"2\",\"uuid\":\"60fb6276-d302-4949-b9eb-9bbc2ca23ce8\",\"cmdEventType\":\"NET.RULEADD\",\"vmguestip\":\"172.30.56.13\",\"protocol\":\"tcp\",\"cidrlist\":\"0.0.0.0\/0\",\"privateport\":\"22\",\"virtualmachineid\":\"9ff71984-2a1c-4bdb-977b-77dc94fe5fd0\",\"response\":\"json\",\"ctxUserId\":\"2\",\"publicport\":\"22\",\"networkid\":\"236386d2-54e1-43fe-9a2a-9f52121bbfcf\",\"ctxStartEventId\":\"9697\",\"id\":\"91\",\"ctxDetails\":\"{\\\"interface com.cloud.network.Network\\\":\\\"236386d2-54e1-43fe-9a2a-9f52121bbfcf\\\",\\\"interface com.cloud.network.rules.FirewallRule\\\":\\\"60fb6276-d302-4949-b9eb-9bbc2ca23ce8\\\",\\\"interface com.cloud.vm.VirtualMachine\\\":\\\"9ff71984-2a1c-4bdb-977b-77dc94fe5fd0\\\",\\\"interface com.cloud.network.IpAddress\\\":\\\"230299bc-723e-4b8d-8e78-02073099fade\\\"}\"}","cmdVersion":0,"completeMsid":null,"created":null,"id":1600,"initMsid":206863092181491,"instanceId":91,"instanceType":"FirewallRule","lastPolled":null,"lastUpdated":null,"processStatus":0,"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"a308b23b-0a6a-40cf-ba53-bdb817460fef"}
2026-09-10 15:11:12,795 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Account [Account [{"accountName":"admin","id":2,"uuid":"47dc9e4c-83f6-11f1-aed8-bc24112069f3"}]] has access to resource.
2026-09-10 15:11:12,795 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Account [Account [{"accountName":"admin","id":2,"uuid":"47dc9e4c-83f6-11f1-aed8-bc24112069f3"}]] has access to resource.
2026-09-10 15:11:12,814 DEBUG [o.a.c.n.t.BasicNetworkTopology] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) APPLYING FIREWALL RULES
2026-09-10 15:11:12,814 DEBUG [o.a.c.n.t.BasicNetworkTopology] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Applying firewall rules in network Network {"id": 242, "name": "tier1", "uuid": "236386d2-54e1-43fe-9a2a-9f52121bbfcf", "networkofferingid": 13}
2026-09-10 15:11:12,817 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Wait time setting on com.cloud.agent.api.routing.SetPortForwardingRulesVpcCommand is 1800 seconds
2026-09-10 15:11:12,817 DEBUG [c.c.a.m.ClusteredAgentAttache] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Seq 13-7910009795522856999: Routed from 206863092181491
2026-09-10 15:11:12,817 DEBUG [c.c.a.t.Request] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Seq 18-7910009795522856999: Sending { Cmd , MgmtId: 206863092181491, via: 18(cloudstack423-kvmhost3), Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.SetPortForwardingRulesVpcCommand":{"rules":[{"dstIp":"172.30.56.13","dstPortRange":["22","22"],"id":"91","srcIp":"192.168.55.85","protocol":"tcp","srcPortRange":["22","22"],"revoked":"false","alreadyAdded":"false","sourceCidrList":["0.0.0.0/0"],"purpose":"PortForwarding","defaultEgressPolicy":"false"}],"accessDetails":{"router.name":"r-163-VM","router.guest.ip":"172.30.56.8","router.ip":"169.254.60.209","zone.network.type":"Advanced"},"wait":"0","bypassHostMaintenance":"false"}}] }
2026-09-10 15:11:13,557 DEBUG [c.c.a.t.Request] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Seq 18-7910009795522856999: Received: { Ans: , MgmtId: 206863092181491, via: 18(cloudstack423-kvmhost3), Ver: v1, Flags: 0, { GroupAnswer } }
2026-09-10 15:11:13,583 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Complete async job-1600, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.FirewallRuleResponse/portforwardingrule/{"id":"60fb6276-d302-4949-b9eb-9bbc2ca23ce8","privateport":"22","privateendport":"22","protocol":"tcp","publicport":"22","publicendport":"22","virtualmachineid":"9ff71984-2a1c-4bdb-977b-77dc94fe5fd0","virtualmachinename":"VM-9ff71984-2a1c-4bdb-977b-77dc94fe5fd0","virtualmachinedisplayname":"VM-9ff71984-2a1c-4bdb-977b-77dc94fe5fd0","ipaddressid":"230299bc-723e-4b8d-8e78-02073099fade","ipaddress":"192.168.55.85","state":"Active","cidrlist":"0.0.0.0/0","tags":[],"vmguestip":"172.30.56.13","networkid":"236386d2-54e1-43fe-9a2a-9f52121bbfcf","networkname":"tier1","fordisplay":"true"}
2026-09-10 15:11:13,583 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Publish async job-1600 complete on message bus
2026-09-10 15:11:13,583 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Wake up jobs related to job-1600
2026-09-10 15:11:13,583 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Update db status for job-1600
2026-09-10 15:11:13,584 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-106:[ctx-1340fe97, job-1600, ctx-9406f6cd]) (logid:a308b23b) Wake up jobs joined with job-1600 and disjoin all subjobs created from job- 1600
2026-09-10 15:11:13,593 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl$5] (API-Job-Executor-106:[ctx-1340fe97, job-1600]) (logid:a308b23b) Done executing org.apache.cloudstack.api.command.user.firewall.CreatePortForwardingRuleCmd for job-1600
2026-09-10 15:11:13,593 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-106:[ctx-1340fe97, job-1600]) (logid:a308b23b) Remove job-1600 from job monitoring
kvm Agent log
root@cloudstack423-kvmhost3:/home/ubuntu# cat /var/log/cloudstack/agent/agent.log |grep -i "logid:a308b23b"
2026-09-10 15:11:12,813 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) (logid:a308b23b) Request:Seq 18-7910009795522856999: { Cmd , MgmtId: 206863092181491, via: 18, Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.SetPortForwardingRulesVpcCommand":{"rules":[{"dstIp":"172.30.56.13","dstPortRange":["22","22"],"id":"91","srcIp":"192.168.55.85","protocol":"tcp","srcPortRange":["22","22"],"revoked":"false","alreadyAdded":"false","sourceCidrList":["0.0.0.0/0"],"purpose":"PortForwarding","defaultEgressPolicy":"false"}],"accessDetails":{"router.guest.ip":"172.30.56.8","router.ip":"169.254.60.209","zone.network.type":"Advanced","router.name":"r-163-VM"},"wait":"0","bypassHostMaintenance":"false"}}] }
2026-09-10 15:11:12,813 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) (logid:a308b23b) Processing command: com.cloud.agent.api.routing.SetPortForwardingRulesVpcCommand
2026-09-10 15:11:12,813 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Transforming com.cloud.agent.api.routing.SetPortForwardingRulesVpcCommand to ConfigItems
2026-09-10 15:11:12,813 DEBUG [virtualnetwork.facade.SetPortForwardingRulesVpcConfigItem] (AgentRequest-Handler-4:[]) (logid:a308b23b) Transformed filename: forwarding_rules.json to: forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c
2026-09-10 15:11:12,813 DEBUG [kvm.resource.LibvirtComputingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Creating file in VR, with ip: 169.254.60.209, file: forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c
2026-09-10 15:11:13,025 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Processing FileConfigItem, copying 271 characters to forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c took 212ms
2026-09-10 15:11:13,025 DEBUG [kvm.resource.LibvirtComputingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Executing command [/usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh update_config.py 169.254.60.209 forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c ].
2026-09-10 15:11:13,511 DEBUG [kvm.resource.LibvirtComputingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Successfully executed process [28698] for command [/usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh update_config.py 169.254.60.209 forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c ].
2026-09-10 15:11:13,511 DEBUG [kvm.resource.LibvirtComputingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Executing script in VR: update_config.py
2026-09-10 15:11:13,511 DEBUG [resource.virtualnetwork.VirtualRoutingResource] (AgentRequest-Handler-4:[]) (logid:a308b23b) Processing ScriptConfigItem, executing update_config.py forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c took 486ms
2026-09-10 15:11:13,511 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-4:[]) (logid:a308b23b) Seq 18-7910009795522856999: { Ans: , MgmtId: 206863092181491, via: 18, Ver: v1, Flags: 0, [{"com.cloud.agent.api.routing.GroupAnswer":{"results":["null - success: Creating file in VR, with ip: 169.254.60.209, file: forwarding_rules.json.53705e62-6d1d-4749-ad45-4764e8d6902c","null - success: iptables: Bad rule (does a matching rule exist in that chain?).
Router log
2026-09-10 15:11:13,260 INFO Address found in DataBag ==> {'add': True, 'broadcast': '169.254.255.255', 'cidr': '169.254.60.209/16', 'device': 'eth0', 'gateway': '', 'netmask': '255.255.0.0', 'network': '169.254.0.0/16', 'nic_dev_id': '0', 'nw_type': 'control', 'one_to_one_nat': False, 'public_ip': '169.254.60.209', 'size': '16', 'source_nat': False}
2026-09-10 15:11:13,260 INFO Address 169.254.60.209/16 on device eth0 already configured
2026-09-10 15:11:13,261 INFO Executing: ip addr show dev eth1
2026-09-10 15:11:13,262 INFO Address found in DataBag ==> {'add': True, 'broadcast': '192.168.55.255', 'cidr': '192.168.55.83/24', 'device': 'eth1', 'first_i_p': True, 'gateway': '192.168.55.1', 'is_private_gateway': False, 'mtu': '1500', 'netmask': '255.255.255.0', 'network': '192.168.55.0/24', 'new_nic': False, 'nic_dev_id': 1, 'nw_type': 'public', 'one_to_one_nat': False, 'public_ip': '192.168.55.83', 'size': '24', 'source_nat': True, 'vif_mac_address': '1e:01:c0:00:00:16'}
2026-09-10 15:11:13,262 INFO Address 192.168.55.83/24 on device eth1 already configured
2026-09-10 15:11:13,262 INFO Adding route table: 101 Table_eth1 to /etc/iproute2/rt_tables if not present
2026-09-10 15:11:13,262 INFO Executing: ip rule show
2026-09-10 15:11:13,264 INFO Executing: ip rule show
2026-09-10 15:11:13,265 INFO Executing: ip link show eth1 | grep ' state '
2026-09-10 15:11:13,267 INFO Executing2: arping -c 1 -I eth1 -A -U -s 192.168.55.83 192.168.55.1
2026-09-10 15:11:13,267 INFO Adding route: dev eth1 table: Table_eth1 network: 192.168.55.1 if not present
2026-09-10 15:11:13,268 INFO Executing: ip route show default via 192.168.55.1 table Table_eth1 proto static
2026-09-10 15:11:13,270 INFO Adding route: dev eth1 table: Table_eth1 network: 172.30.56.0/28 if not present
2026-09-10 15:11:13,271 INFO Executing: ip route show 172.30.56.0/28 table Table_eth1 proto static
2026-09-10 15:11:13,274 INFO Executing: ip rule show
2026-09-10 15:11:13,277 INFO Adding route: dev eth1 table: Table_eth1 network: 192.168.55.0/24 if not present
2026-09-10 15:11:13,277 INFO Executing: ip route show 192.168.55.0/24 table Table_eth1 proto static
2026-09-10 15:11:13,279 INFO Executing: sudo ip route flush cache
2026-09-10 15:11:13,284 INFO Checking if default IPv4 route is present
2026-09-10 15:11:13,284 INFO Executing: ip -4 route list 0/0
2026-09-10 15:11:13,285 INFO Default route found: default via 192.168.55.1 dev eth1
2026-09-10 15:11:13,285 INFO Executing: ip addr show dev eth2
2026-09-10 15:11:13,286 INFO Address found in DataBag ==> {'add': True, 'broadcast': '172.30.56.15', 'cidr': '172.30.56.8/28', 'device': 'eth2', 'gateway': '172.30.56.8', 'mtu': '1500', 'netmask': '255.255.255.240', 'network': '172.30.56.0/28', 'nic_dev_id': '2', 'nw_type': 'guest', 'one_to_one_nat': False, 'public_ip': '172.30.56.8', 'size': '28', 'source_nat': False}
2026-09-10 15:11:13,287 INFO Address 172.30.56.8/28 on device eth2 already configured
2026-09-10 15:11:13,287 INFO Adding route table: 102 Table_eth2 to /etc/iproute2/rt_tables if not present
2026-09-10 15:11:13,287 INFO Executing: ip rule show
2026-09-10 15:11:13,288 INFO Executing: ip rule show
2026-09-10 15:11:13,289 INFO Executing: ip link show eth2 | grep ' state '
2026-09-10 15:11:13,291 INFO Executing2: arping -c 1 -I eth2 -A -U -s 172.30.56.8 172.30.56.8
2026-09-10 15:11:13,292 INFO Adding route: dev eth2 table: Table_eth2 network: 172.30.56.0/28 if not present
2026-09-10 15:11:13,292 INFO Executing: ip route show 172.30.56.0/28 table Table_eth2 proto static
2026-09-10 15:11:13,294 INFO Executing: sudo ip route flush cache
2026-09-10 15:11:13,298 ERROR Not able to setup source-nat for a regular router yet
2026-09-10 15:11:13,298 INFO Not making dns publicly available
2026-09-10 15:11:13,299 INFO Executing: systemctl start cloud-password-server@172.30.56.8
2026-09-10 15:11:13,303 INFO Service cloud-password-server@172.30.56.8 start
2026-09-10 15:11:13,303 INFO Flush all IPv6 ACL rules
2026-09-10 15:11:13,303 INFO Executing: nft list tables ip6 | grep ip6_acl
2026-09-10 15:11:13,307 ERROR Command 'nft list tables ip6 | grep ip6_acl' returned non-zero exit status 1.
2026-09-10 15:11:13,307 INFO Executing: iptables-save -t mangle
2026-09-10 15:11:13,309 INFO Executing: iptables-save | grep '^:FW_EGRESS_RULES' || iptables -t filter -N FW_EGRESS_RULES
2026-09-10 15:11:13,311 INFO Executing: iptables-save | grep '^-A FW_EGRESS_RULES -j ACCEPT$' | sed 's/^-A/iptables -t filter -D/g' | bash
2026-09-10 15:11:13,315 INFO Executing: iptables -F FW_EGRESS_RULES
2026-09-10 15:11:13,316 INFO Executing: ipset -L | grep Name: | awk {'print $2'} | ipset flush
2026-09-10 15:11:13,319 INFO Executing: ipset -L | grep Name: | awk {'print $2'} | ipset destroy
2026-09-10 15:11:13,322 INFO Executing: iptables-save -t mangle
2026-09-10 15:11:13,324 INFO Flush all IPv6 firewall rules
2026-09-10 15:11:13,324 INFO Executing: nft list tables ip6 | grep ip6_firewall
2026-09-10 15:11:13,327 ERROR Command 'nft list tables ip6 | grep ip6_firewall' returned non-zero exit status 1.
2026-09-10 15:11:13,327 INFO Processing IPv6 firewall rules {'id': 'ipv6firewallrules'}; []
2026-09-10 15:11:13,327 INFO Executing: iptables-save
2026-09-10 15:11:13,330 INFO Add: rule=-A VPN_STATS_eth1 -o eth1 -m mark --mark 0x525/0xffffffff table=mangle
2026-09-10 15:11:13,330 INFO Executing2: iptables -t mangle -A VPN_STATS_eth1 -o eth1 -m mark --mark 0x525/0xffffffff
2026-09-10 15:11:13,331 INFO Add: rule=-A VPN_STATS_eth1 -i eth1 -m mark --mark 0x524/0xffffffff table=mangle
2026-09-10 15:11:13,331 INFO Executing2: iptables -t mangle -A VPN_STATS_eth1 -i eth1 -m mark --mark 0x524/0xffffffff
2026-09-10 15:11:13,332 INFO Add: rule=-A NETWORK_STATS_eth1 -s 172.30.56.7/24 -o eth1 table=filter
2026-09-10 15:11:13,332 INFO Executing2: iptables -t filter -A NETWORK_STATS_eth1 -s 172.30.56.7/24 -o eth1
2026-09-10 15:11:13,334 INFO Add: rule=-A NETWORK_STATS_eth1 -d 172.30.56.7/24 -i eth1 table=filter
2026-09-10 15:11:13,334 INFO Executing2: iptables -t filter -A NETWORK_STATS_eth1 -d 172.30.56.7/24 -i eth1
2026-09-10 15:11:13,335 INFO Add: rule=-A NETWORK_STATS_eth1 ! -s 172.30.56.7/24 -o eth1 -p tcp table=filter
2026-09-10 15:11:13,335 INFO Executing2: iptables -t filter -A NETWORK_STATS_eth1 ! -s 172.30.56.7/24 -o eth1 -p tcp
2026-09-10 15:11:13,336 INFO Add: rule=-A NETWORK_STATS_eth1 ! -d 172.30.56.7/24 -i eth1 -p tcp table=filter
2026-09-10 15:11:13,336 INFO Executing2: iptables -t filter -A NETWORK_STATS_eth1 ! -d 172.30.56.7/24 -i eth1 -p tcp
2026-09-10 15:11:13,337 INFO Add: rule=-P INPUT DROP table=filter
2026-09-10 15:11:13,338 INFO Executing2: iptables -t filter -P INPUT DROP
2026-09-10 15:11:13,343 INFO Add: rule=-P FORWARD DROP table=filter
2026-09-10 15:11:13,343 INFO Executing2: iptables -t filter -P FORWARD DROP
2026-09-10 15:11:13,355 INFO Add: rule=-A FORWARD -s 172.30.56.7/24 ! -d 172.30.56.7/24 -j ACCEPT table=filter
2026-09-10 15:11:13,355 INFO Executing2: iptables -t filter -I FORWARD 3 -s 172.30.56.7/24 ! -d 172.30.56.7/24 -j ACCEPT
2026-09-10 15:11:13,357 INFO Add: rule=-A ACL_INBOUND_eth2 -p all -s 0.0.0.0/0 -j ACCEPT table=filter
2026-09-10 15:11:13,357 INFO Executing2: iptables -t filter -I ACL_INBOUND_eth2 4 -p all -s 0.0.0.0/0 -j ACCEPT
2026-09-10 15:11:13,358 INFO Add: rule=-A ACL_OUTBOUND_eth2 -p all -d 0.0.0.0/0 -j ACCEPT table=mangle
2026-09-10 15:11:13,358 INFO Executing2: iptables -t mangle -I ACL_OUTBOUND_eth2 3 -p all -d 0.0.0.0/0 -j ACCEPT
2026-09-10 15:11:13,360 INFO Add: rule= -A FIREWALL_192.168.55.85 -s 0.0.0.0/02 -p tcp -m tcp --dport 22 -j RETURN table=mangle
2026-09-10 15:11:13,360 INFO Executing2: iptables -t mangle -I FIREWALL_192.168.55.85 -s 0.0.0.0/02 -p tcp -m tcp --dport 22 -j RETURN
2026-09-10 15:11:13,361 INFO Add: rule=-A PREROUTING -s 0.0.0.0/0 -d 192.168.55.85/32 -m tcp -p tcp --dport 22 -j DNAT --to-destination 172.30.56.13:22 table=nat
2026-09-10 15:11:13,361 INFO Executing2: iptables -t nat -A PREROUTING -s 0.0.0.0/0 -d 192.168.55.85/32 -m tcp -p tcp --dport 22 -j DNAT --to-destination 172.30.56.13:22
2026-09-10 15:11:13,362 INFO Add: rule=-j SNAT --to-source 172.30.56.8 -A POSTROUTING -s 172.30.56.0/28 -d 172.30.56.13/32 -o eth2 -p tcp -m tcp --dport 22 table=nat
2026-09-10 15:11:13,363 INFO Executing2: iptables -t nat -j SNAT --to-source 172.30.56.8 -A POSTROUTING -s 172.30.56.0/28 -d 172.30.56.13/32 -o eth2 -p tcp -m tcp --dport 22
2026-09-10 15:11:13,364 INFO Add: rule=-A OUTPUT -s 0.0.0.0/0 -d 192.168.55.85/32 -m tcp -p tcp --dport 22 -j DNAT --to-destination 172.30.56.13:22 table=nat
2026-09-10 15:11:13,364 INFO Executing2: iptables -t nat -A OUTPUT -s 0.0.0.0/0 -d 192.168.55.85/32 -m tcp -p tcp --dport 22 -j DNAT --to-destination 172.30.56.13:22
2026-09-10 15:11:13,365 INFO Executing: iptables -t mangle -D ACL_OUTBOUND_eth2 -j ACCEPT
2026-09-10 15:11:13,371 INFO Delete rule -D ACL_OUTBOUND_eth2 -j ACCEPT from table mangle
2026-09-10 15:11:13,371 INFO Executing: iptables -t mangle -D FIREWALL_192.168.55.85 -s 0.0.0.0/2 -p tcp -m tcp --dport 22 -j RETURN
2026-09-10 15:11:13,383 INFO Delete rule -D FIREWALL_192.168.55.85 -s 0.0.0.0/2 -p tcp -m tcp --dport 22 -j RETURN from table mangle
2026-09-10 15:11:13,383 INFO Executing: iptables -t mangle -D VPN_STATS_eth1 -m mark -o eth1 --mark 0x525
2026-09-10 15:11:13,395 INFO Delete rule -D VPN_STATS_eth1 -m mark -o eth1 --mark 0x525 from table mangle
2026-09-10 15:11:13,395 INFO Executing: iptables -t mangle -D VPN_STATS_eth1 -i eth1 -m mark --mark 0x524
2026-09-10 15:11:13,407 INFO Delete rule -D VPN_STATS_eth1 -i eth1 -m mark --mark 0x524 from table mangle
2026-09-10 15:11:13,407 INFO Executing: iptables -t filter -D FORWARD -s 172.30.56.0/24 ! -d 172.30.56.0/24 -j ACCEPT
2026-09-10 15:11:13,419 INFO Delete rule -D FORWARD -s 172.30.56.0/24 ! -d 172.30.56.0/24 -j ACCEPT from table filter
2026-09-10 15:11:13,419 INFO Executing: iptables -t filter -D ACL_INBOUND_eth2 -j ACCEPT
2026-09-10 15:11:13,431 INFO Delete rule -D ACL_INBOUND_eth2 -j ACCEPT from table filter
2026-09-10 15:11:13,431 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1
2026-09-10 15:11:13,433 ERROR Command 'iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1' returned non-zero exit status 1.
2026-09-10 15:11:13,433 INFO Delete rule -D NETWORK_STATS_eth1 -p tcp -o eth1 from table filter
2026-09-10 15:11:13,433 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1
2026-09-10 15:11:13,434 ERROR Command 'iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1' returned non-zero exit status 1.
2026-09-10 15:11:13,434 INFO Delete rule -D NETWORK_STATS_eth1 -p tcp -o eth1 from table filter
2026-09-10 15:11:13,434 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 -s 172.30.56.0/24 -o eth1
2026-09-10 15:11:13,447 INFO Delete rule -D NETWORK_STATS_eth1 -s 172.30.56.0/24 -o eth1 from table filter
2026-09-10 15:11:13,447 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 -d 172.30.56.0/24 -i eth1
2026-09-10 15:11:13,459 INFO Delete rule -D NETWORK_STATS_eth1 -d 172.30.56.0/24 -i eth1 from table filter
2026-09-10 15:11:13,459 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1
2026-09-10 15:11:13,461 ERROR Command 'iptables -t filter -D NETWORK_STATS_eth1 -p tcp -o eth1' returned non-zero exit status 1.
2026-09-10 15:11:13,461 INFO Delete rule -D NETWORK_STATS_eth1 -p tcp -o eth1 from table filter
2026-09-10 15:11:13,461 INFO Executing: iptables -t filter -D NETWORK_STATS_eth1 ! -d 172.30.56.0/24 -i eth1 -p tcp
2026-09-10 15:11:13,471 INFO Delete rule -D NETWORK_STATS_eth1 ! -d 172.30.56.0/24 -i eth1 -p tcp from table filter
2026-09-10 15:11:13,471 INFO Configuring nftables IPv4 firewall rules []
2026-09-10 15:11:13,472 INFO Executing: iptables-save
2026-09-10 15:11:13,473 INFO Configuring nftables IPv4 ACL rules []
2026-09-10 15:11:13,474 INFO Executing: iptables-save
2026-09-10 15:11:13,476 INFO Configuring nftables IPv6 ACL rules []
2026-09-10 15:11:13,476 INFO Executing: iptables-save
2026-09-10 15:11:13,478 INFO Configuring nftables IPv6 firewall rules []
2026-09-10 15:11:13,478 INFO Executing: iptables-save
2026-09-10 15:11:13,480 INFO Executing: iptables-save
2026-09-10 15:11:13,482 INFO Executing: ip6tables-save
2026-09-10 15:11:13,483 INFO Executing: nft list ruleset
2026-09-10 15:11:13,489 INFO Executing: systemctl stop conntrackd
2026-09-10 15:11:13,494 INFO Service conntrackd stop
2026-09-10 15:11:13,495 INFO Executing: systemctl stop keepalived
2026-09-10 15:11:13,500 INFO Service keepalived stop
2026-09-10 15:11:13,500 INFO Executing: mount
iptables
root@r-184-VM:~# iptables -t nat -S | grep -E 'DNAT|SNAT'
-A PREROUTING -d 192.168.55.84/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 172.30.45.7:22
-A OUTPUT -d 192.168.55.84/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 172.30.45.7:22
-A POSTROUTING -o eth1 -j SNAT --to-source 192.168.55.83
-A POSTROUTING -s 172.30.45.0/28 -d 172.30.45.7/32 -o eth2 -p tcp -m tcp --dport 22 -j SNAT --to-source 172.30.45.2
root@r-184-VM:~# iptables-save | grep -i firewall
:FIREWALL_192.168.55.84 - [0:0]
-A PREROUTING -d 192.168.55.84/32 -j FIREWALL_192.168.55.84
-A FIREWALL_192.168.55.84 -p tcp -m tcp --dport 22 -j RETURN
-A FIREWALL_192.168.55.84 -m state --state RELATED,ESTABLISHED -j RETURN
-A FIREWALL_192.168.55.84 -j DROP
What to do about it?
Workaround
Restart the vpc with cleanup; applies the rule correctly and the user is able to ssh
User should be able to ssh when there is firewall and PF rule present in the vpc
problem
CreatePortForwardingRuleCmd on a VPC tier doesn't trigger ipassoc ( firewall and conserve mode on in vpc offering)
versions
ACS 4.23
The steps to reproduce the bug
4.23 introduced the conserve mode and firewall feature in vpc
https://docs.cloudstack.apache.org/en/4.23.0.0/adminguide/networking/virtual_private_cloud_config.html
logs
Management server log
kvm Agent log
Router log
iptables
What to do about it?
Workaround
Restart the vpc with cleanup; applies the rule correctly and the user is able to ssh
User should be able to ssh when there is firewall and PF rule present in the vpc